It seems to depend on how the image was made in the first place, & what its current background is set to.
Testing with the example given in comments I see the same behaviour as the OP when using Preview.
Even in Photoshop the same thing happens... unless I turn the image into an unlocked layer first. After that, erasing will then erase to transparent.
I've tested with other images made in photoshop with transparency, then saved as jpg. I also checked converting the jpg to png first, before testing.
It seems Preview is not the only app which will knock out to a predetermined colour, Photoshop will do the same, unless first converted to a separate layer. [In Photoshop you get the choice of which colour, but 'transparent' isn't an option.]
I'd say therefore, it's not a bug, it's intended behaviour; a limitation of a single-layered image.
I see no way round this except to use a dedicated graphics package, like Photoshop or Gimp.