The reason for having these folders instead of a more simple /tmp that you might know from other systems is to prevent a number of security issues. Mainly some programs create files/folders/pipes in /tmp that could be used to exploit that program or user data. Therefore these programs need to ensure that the file permissions on those files are such that other user's programs cannot access them.
The problem is that this is sometimes forgotten or ignored, and this creates security holes. This is not only the case on macOS, but also on Linux that you mention as your comparison. There exist similar solutions on Linux to this problem, but contrary to macOS, they're usually not installed by default on a home user setup. On Linux these solutions are known as "polyinstantiated /tmp directory" and "PrivateTmp".
The solution on macOS is to have a seperate temporary files folder for each user. This way one user cannot interfere with other user's temporary folders (due to name clashes or intentionally).
The placement in /var/folders is most probably just to keep it away from /Users, which would otherwise be a logical place to store something for each user. The reasoning is that this folder contains things that are temporary of nature, and therefore you do not want it in /Users, which could be a network share on a server, and most probably included in backups, etc.
The odd name "k7jkgy2s3_99p4vhlbx3" you see is actually just generated from the unix uid (user id, 32-bit) and the users uuid (universally unique identifier, 128-bit). Those two are concatenated and then converted into a file name by mapping each set of 5 bits to a character from the set '0123456789_bcdfghjklmnpqrstvwxyz'. This makes it harder for programs to intentionally opening up file paths for other users, as they would need to know this name in advance.
The folders "pd" and "zz" that you call TLDs are simply the first two characters of the above mentioned encoding. The idea is that if you have a large corporate system then you wouldn't want to have thousands of folders (one for each user) inside the same folder. Instead you have fewer folders that are combinations of two characters (like pd, zz, bc, etc.) - and in each of those you then have for example hundreds of folders for those users.
The folders below that again, "C, T, 0" are just identifiers to specify that the contents of those folders are "caches", "temporary files" and "user files" accordingly.