I plan to use Time Machine to back up by data.
If the backup is not encrypted, can someone with access to the TimeMachine disk access my keychain passwords? Do they at least need to know my system login password in order to be able to do this?
Keychain files are password protected, your login password is required to access their content. In the file itself only passwords and the content of secure notes are encrypted so a potential attacker could at least read the URLs and the associated login names for web accounts etc.
For details about Keychain see Wikipeda and the pages referenced from there.