Apple says to remove an item from the dock in Yosemite, drag it from the dock until you see a puff of smoke. I drag it from the dock, it has a tag called "remove" but it never goes up in smoke.It just returns to the dock.
Even though the items appeared not to be open, they were open. By holding down the right click on the icon, a drop down menu appeared with the option to "quit." Select that item, and the icon is removed.
This can be caused by a couple things.
Start with Repairing the Disk permissions through Disk Utility. Open Disk Utility, select the drive from the list on the left, and click Repair Disk Permissions. When it is finished, restart your computer. According to Apple, repairing the permissions only applies to "files installed by the Mac OS X Installer, Software Update, or an Apple software installer. It doesn’t repair permissions for your documents, your home folder, or third-party applications."
Rebooting may fix the issue, but temporarily. If so, it could be caused by some third party application interfering with the Docks normal function. Disable startup items for your account and see if the issue persists. You can also try and create a new, clean User account and see if the problem is system wide, or only specific to your account.
I thought this may be late but it can help you to sort it out. Open Terminal in Utilities and use this command line:
defaults delete com.apple.dock; killall Dock