Why are some of the file names in the /Library/Fonts folder, e.g. AppleGothic, missing from Font Book's (top level) font name list?
Why are some of the (top level) font names in Font Book, e.g. Arial, missing from the /Library/Fonts folder?
There are some font names that appear both in the /Library/Fonts folder as well as in Font Book, but with different spacing, e.g. AmericanTypewriter (/Library/Fonts) vs. American Typewriter (Font Book). Why?
Are there any other differences between the filenames in /Library/Fonts and the font names in Font Book that I should be aware of?
This all stems from how font files have a file name as well as a family name and a full name stored inside the file. The file name is what is seen in Finder when looking at /Library/Fonts, whereas the family name is what is shown at the top level in Font Book (with the added items for the full name shown when you click on the triangle). You can find the full name of a font file from Finder using Get Info.
This screenshot shows the answer to your point 3. The file is called AmericanTypewriter, but it could be called anything, because the actual font within the file is called American Typewriter and won't be changed by renaming the file.
To find a font file given the font listing in Font Book, right click on the font and choose Show in Finder. This will select all the font files which are used to make up this font face.
AppleGothic is not missing from the list of fonts in Font Book, it's just at the bottom, out of alphabetical order.