The examples you've used in your question can be explained by the use of the SSD in the MacBook Air.
In March 2014 I installed a hybrid drive (what Apple calls a 'Fusion' drive) into a late 2008 MacBook Pro. The results of doing this compared to the standard HDD that was installed previously were significant!
Now, remember this was just a hybrid drive NOT an SSD. My tests showed that:
- bootup time was 16 seconds faster
- login time was 6 seconds faster
- launching MS Word was 17 seconds faster
I then tested this against a mid-2011 iMac with a traditional 7200rpm drive and the MacBook Pro was faster! Remember again this was just a hybrid drive NOT an SSD in the MacBook Pro.
One thing you should notice on the iMac though is that the first time you launch an application after booting up it will take longer than subsequent launches (unless you've rebooted again).
So yes, the SSD will explain the difference in the examples you gave, but there will be other tasks that your iMac will perform faster because they rely more on processing power rather than disk read/write times.
Whether you should install an SSD is a matter for you. It'll improve speed generally, but how much will depend on what you use your iMac for.
[EDIT]
I found the full results of my testing for the hybrid drive in my late 2008 MacBook Pro. See the table below:

As you can see, the response times also improved by doing a reformat and clean install on the original HDD. However, since you've got a late 2015 iMac I don't imagine you've had enough time to weigh the system down with unnecessary software etc.
The point I guess I'm trying to make is that the answer to your question Is the HDD/SSD the main cause of this? is yes for the examples you gave, but if you have other real life examples that would be good for some context.