I am playing a game called Bookworm on the Mac, and it starts as a full screen game, and the only way seems to be from its Options to choose not to run in full screen. What if a game doesn't provide such option, is there a way to make it become a window? (force the game to run in window mode instead of full screen).
Yes and No. The game has to support the ability to run in windowed mode. If it doesn't support it, then it will not be able to do it.
Now, when it comes to games that don't have the option in an preference menu, they can sometimes still support the feature, they just don't advertise it.
There are a few games that require you to edit a preference file to enable/disable certain things. Usually contained inside the game icon, right clicking on the game icon and hitting "Show Package Contents" will open the game's directory and show you the files that make the game. Be very careful when doing this because editing the wrong thing can break the game. Usually if this option exists then it will be somewhere on the internet and you will be able to find out more about it by doing a search for that game.
One such game that supports this is Cave Story+, editing the preference file can enable certain game modes and unlock features.
Other games will work with a shortcut when running, usually:
opt/alt + enter
opt/alt + f
cmd + f
cmd + opt + f
cmd + opt + enter
Now, if none of these are viable options, then you won't be able to do it.
I hope that helps clarify things.