Outlook is (for comparison purposes) a glorified IMAP serverclient. EG it keeps everything on the server and Outlook is just viewing and changing data on the server.
So what you need to do is to tell Outlook to stop looking on the server for changes, which it does every few seconds.
Fortunately there has been an option in Outlook for a long time to allow you to do this.
- With Outlook active click on the Outlook menu
- Select "Work Offline"
- Enjoy your notification free existence
When you reconnect to your VPN you just have to go back to that same menu and select Work Online. With no dipping into network or notification settings, easy peasy.