The mouse is a capacitive touch, so you need to choose a thinner glove material or one that’s designed for capacitive touch screens. Many of these exist on the market for outdoor / cold weather use, but I’m not aware of an indoor version that I can recommend.
Here is what might be suitable to consider: Sibba Touch Screen Moisturizing Gloves
Go with a (very if possible) thin glove for now is my advice.