I'm considering getting an iPod Touch for my car exclusively. Will iCloud allow the 8 GB version to play my entire library?
iCloud on it's own will not provide any library functions, the closest chance you have is if your car is in range of your Wifi network and you occasionally Wifi Sync it from iTunes.
Otherwise, your other opinion is iTunes Match. You can use this to allow your iPod to download any songs in your library (with just a few caveats) for playback on the iPod touch. Note that you can download a song by tapping the cloud icon, but tapping normally on an un-downloaded song will download it and stream/play it at the same time.
This means that you have limited effectiveness of updating your iPod touch, as you will only be able to do so when near a Wifi Access Point. You will always have access to downloaded files no matter your network state, of course.