I think the key is that iOS gives you control over two "types" of volume: sound volume (for music, videos, etc.) and ringer volume (for notifications, ringtones, etc.). It seems you have indeed muted the sound volume, but your ringer volume is still up. Depending on your settings, one of the following should completely mute your iPad (including notifications):
- Slide the side switch right above the volume buttons on the right side of your iPad.
- Swipe up from the bottom of your screen to reveal the Control Center. (On newer devices, you access the Control Center by swiping down from the top of the screen). Tap the mute button, which looks like a bell.
You can set the alert volume in Settings > Sounds, if you'd like to turn down the volume or turn it off completely. If you want, you can also enable the switch to allow you to change the alert volume with your volume buttons when no music is playing.