Two possible causes for your troubles as I see it with the information provided is that either the bless command for the SSD is corrupted or your NVRAM is corrupted.
Have you tried reseting the NVRAM/PRAM? This is done by holding down opt + cmd + p + r right after you power the Mac. Hold these four keys down until you hear the boot chime for the third time after which you can let go and your system will have cleared out what it boots to by default.
That is only the first portion of the fix because it will boot quicker now (or should) but you need to tell your system to boot to the SSD. Go into System Preferences and then Startup Disk as Kent suggest and highlight the SSD. At this point I even go far enough to tell Startup Disk to restart... This will test it immediately and you can let us now if that resolved the troubles you are having.
If not then you might have a damaged OS or some corrupted settings that are making your restarts unclean. I love a program called Onyx for clearing caches and Saved Application States. Try that out if issue remains and holler back if need be. http://www.titanium.free.fr/downloadonyx.php