In iTunes on your computer, you will need to select what music you want to sync. When the phone is plugged into iTunes and the phone is selected in the sidebar (you may need to select the menu item "View" -> "Show Sidebar"), one of the options beside the "Summary" of the phone is for "Music" where you can set exactly which playlists, artists, albums, and genres get synced. I find it convenient to have at least one playlist called something like "for-my-iphone" that I can conveniently add any individual tracks that I want synced to the phone, in addition to any of the other possible selection methods.
Incidentally, on the "Summary" page, it is probably worthwhile selecting the "convert higher bitrate songs to 128 kbps" in order to save a lot of room on your iDevice. With most headphones and listening environments it is almost impossible to notice the different sound quality. Checking this box might make the first sync a lot longer if the system needs to do a bunch of music conversion, but subsequent syncs will not be noticeably slower in my experience.