No, it is not usual to experience "really slow performance" on MacOS Extended (Journaled, Encrypted) file systems (also known as HFS+ encrypted). That configuration is extremely widely used, so any intrinsic problems here would have been reported in huge numbers.
I have been using Git-LFS on such a file system for a very long time, and never experience any such slowness. I know that many of my fellow developers do not have the problem either.
I.e. this is not a general problem for everyone, but instead seems to be a local problem with your computer.
I would use regular file system diagnostics to check your file system. Use Disk Utility or fsck on the command line. Also checks your system logs to see if the OS reports any I/O errors or timeouts during these slow performing commands.
As you mentioned it is external harddrive, you might also have an interface problem with your cabling or similar. I would try creating new large files (unrelated files) and see if they are also slow.
Also check the number of files inside the directory you're cd'ing into to see if it is a very large number - that could also potentially be a cause.