I tried using DoubleCommand, but it doesn't let you hold down caps lock, and pressing it causes caps lock light to be enabled/disabled. Ideally I want caps lock to never be enabled and for the key to behave exactly as though it were the delete key.
Another problem that DoubleCommand has is that every second press just toggles the light, and doesn't delete a character.