Although Safari has a setting for a default download folder, it always defaults to the last folder I uploaded something from when downloading via "save as". This is very annoying for obvious reasons: I rarely want to download something to the last opened folder and I rarely remember what that folder was. Is there a way to make it not do this?
I think the whole idea is that any Save is to Downloads, any Save as… is to the folder of your choice - that by default is the last place you used.
I guess this means that saving an image or a linked file always gives you the choice, but saving a page as webarchive doesn't.
I could recommend Default Folder X, which I've had on every Mac I've owned since before they added the X. Amongst other things, it remembers a list of favourites & recents , plus you can click through any app in the way to any open folder on the desktop [even if it's in a different Space, if you set that pref].
Files that are automatically downloaded should go directly to the Downloads folder. You can also right-click on a link, and select "Download Linked File", which will also go directly to the Downloads folder. Selecting "Download Linked File As" will bring up a file dialog.
Right-clicking on an image will bring up a similar contextual menu.
"Save Image To Downloads" will obviously go directly to the Downloads folder; "Save Image As..." will bring up a file dialog.
If you're using Save As from the File menu, then that is for web pages (or archives of a web page), and that will bring up a file dialog.
In all cases where a file dialog appears, you should be able to see the default location, and navigate somewhere else if it's not what you want.
You should see the Finder 'Sidebar' in the file dialog, where you can select the Downloads folder.