The sorting in Finder works whenever I open Finder. However, it behaves quite strangely when I am using another application. Here is an example:
Example: When using Chrome, upon hitting ⌘+S, I am saving some pdf into my desired folder:
Notice the down arrow on the column head of "PDF documents" indicates that the files in this folder are sorted by their names in a descending order (Z to A). However, apparently, the pdf files in the "Saving as" window are not sorted as they should be.
After clicking that the arrow twice, the sorting is finally behaving as it should be by sorting the pdf files by name in a descending order:
This doesn't only apply while "Saving as", it also happens while I am opening some file from another application (not only Chrome, some other apps like Text editors, system tool like Automator). Initially the pop up Finder window does not sort the files as the little down/up arrow indicator shows, I have to manually clicking the arrow twice to get the sorting I want.
PS. I don't know if it matters, but I have used the trick here to make the folder showing first in Finder by modifying Finder's plist file. I am not sure if this results the strange behavior of Finder.