You could look into VeraCrypt which is the open-source successor to TrueCrypt and exists for Windows, OS X, and Linux. It can also open existing TrueCrypt containers. https://veracrypt.codeplex.com
Like TrueCrypt, VeraCrypt can create an encrypted CONTAINER FILE which is like an DMG or ISO image file and can be copied, so it would fit on a FAT32-formatted USB drive as long as the file size remains below 4 GB, as FAT32 cannot contain individual files larger than that. For this a container file you can choose between FAT, exFAT, and OS X Extended as the internal format. For maximum compatibility between Windows and OS X, FAT is the right choice. ExFAT can take files larger than 4 GB and is supported by Windows XP, but it is not supported in OS X before 10.6.5 Snow Leopard.
Finally, you can also use VeraCrypt to create an ENCRYPTED DRIVE VOLUME, so that the whole USB drive or a partition itself is encrypted. However, VeraCrypt doesn't encrypt existing files directly. First you create an empty encrypted container file or drive volume, then you copy your files to it.