I actually experienced this on my '12 MacBook air as well and it was driving me crazy so I think I've figured it out without having to buy any apps like smartsleep, etc.
So first off you need to look at how the Mac operates with its settings. The pmset command from Terminal shows and controls the settings. Don't change anything until you read the man page. Enter
pmset -g in Terminal to see your current settings.
pmset -g | grep hibernate mode
pmset -g | grep standby
Those are the defaults and you should read the full details on Apple's Man Page.
But basically what that means is after 4200s (70min) of regular sleep ("standby" on Windows), it goes into deep sleep ("hibernate" on Windows). From testing, wake from deep sleep takes about 3-4 seconds or up to 10 seconds depending on what you had open (all SSD times of course).
Now you can increase this time which means it remains in standby (aka memory is still powered). I have mine set to 42000s (700min) so that it covers most of my inactive scenarios (sleep, travel etc) and only hibernates on an errant weekend or two where I don't turn my computer on. Mac OS still creates the hibernate file anyways so you don't risk losing data. I have yet to test its affect on battery drain but maybe someone here can do that.
To set the standbydelay to 42000 seconds (11.66 hours), in your Terminal, type:
sudo pmset -a standbydelay 42000
and enter your password.
Now if you'd rather have it based on battery percentage there's a neat loophole. You see Power Nap basically also means that you have to keep standby active as obviously waking from Hibernate is inefficient. So when you check Power Nap on battery you are essentially keeping the computer on Standby until you hit 30% and then it will hibernate. This is because Apple set up Power Nap to do this so you can use this mechanism as well.
I've tried both and now my computer wakes quickly all the time.