I have a Bluetooth keyboard (the Microsoft Surface Ergonomic) paired to OS X. When I boot into Windows, the only way I've found to get it to work is to manually put it into pairing mode again and pair it through the Windows Add a Bluetooth Device option. But then when I reboot back into OS X, the keyboard is disconnected and I have to re-pair again. Is there any way to keep it paired to both?
No, this is not possible.
Looking at this from the perspective your your keyboard, it believes it's connecting to two different machines. When a pairing request is made, due to security, a "key" is generated at that time. The keyboard and host exchange it - then "agree" it's genuine and proceed to pair.
When you switch operating systems, say to macOS now, that OS has never seen your keyboard so when you attempt to pair, it doesn't work as there's no key. So, you pair it and a new key gets generated, agreed that it's genuine and you continue your merry way, keyboarding happily until you switch OSes again.
Now, the key your first OS had doesn't match what it expects from the keyboard and the whole pairing dance has to be repeated. This is because your keyboard only supports a single pairing. What you need is a keyboard that supports multiple devices like the Logitech MX series of products where it will support 3 different devices. There are many of these out there besides Logitech (I personally use a Keychron K8), but some are definitely better than others at the pairing aspect, so be wary of that when selecting one.