Thank you for the comments. I tried some more, and I figured it out.
When I pasted the Unicode symbol, "Apple Color Emoji" would be automatically set (even when choosing "Match Destination Formatting" or "Keep Text Only" from the paste options). When I now tried to select the symbol to change the font (by either positioning the cursor left or right from it and then holding shift while pressing the right/left button on the keyboard once, or by clicking on it and dragging the cursor with the mouse while pressing the left mouse button), it apparently would select not only the symbol itself, but also some whitespace after it (see image). With that selection, I wasn't able to choose a different font.
But when I would adjust the selection to exclude this superfluous whitespace part, it would work! It also works when including other text in the selection, but in my case it was in its own text box with no other text, so I didn't notice.
I feel silly now, but when I first ran into this problem I paid no mind to that whitespace in the selection (and I wouldn't have suspected it could cause a problem). But maybe someone else will have the same issue, who knows.