I use Fantastical for my calendar and thus don't want notifications from Calendar.app since they're redundant with Fantastical's notifications. As you would assume I turned off Calendar.app's notifications but they keep turning back on after a certain period of time. I believe this happens on more than one computer of mine as well.
Every single time I go back into System Preferences and disable the notifications for the app and they turn back on after maybe a couple weeks. This has been a cat and mouse game for the last year or so.
Someone has a similar issue with Messages but I'm not sure it's a similar fix: Notifications for Messages keep turning back on in Yosemite