You have to enter the command du -sh * instead of ls to get the actual size of the folders in your current directory (the sum of the files size they contain).
The command ls shows file size, but it has a different meaning for folders. It actually shows the size allocated for the directory to store metadata of the files it contains.