Apple magic mice and trackpads are all branded as Multi-Touch devices. Just like the iPhone and iOS based devices, there are dozens of gestures with [even more gestures in the newer devices.][1] 

 - https://support.apple.com/en-us/HT204895

If a single touch device (or multi touch that tracks fewer digits or less gestures) is what you ultimately want, that would explain all the confusion and need to learn and then retrain your digits how to not trigger any of the many ways gesture recognition will foil your intentions. Even if you disable most of the gestures in settings, the hardware is still tracking all those touch points even if the OS ignores some the hardware recognizes. 

The mental model needed to move a pointer with a different surface handling the mouse down/up control is very different than a multi touch mental model of input. Just like those of us that learned to drive with a manual transmission, something is missing, even when we find vehicles shift paddles on the steering wheel. The old design allows behaviors the new cannot, despite the other new things multi touch allows. 


  [1]: https://support.apple.com/guide/ipad/trackpad-gestures-ipad66ce6358/ipados