If you can not eject, then the next best thing to do is unmount the volume using the Disk Utility application. Once unmounted you can remove the device from the USB port.
For example, I have an mp3 player plugged in to my USB port. The volume has the label PEARL. If I highlight the volume the Unmount button will appear.
If I click on Unmount, the button will change to Mount. You will also notice the volume is no longer appears in the Finder. I can now unplug the mp3 player.
