Create a silent ringtone with GarageBand. (This assumes you're on a Mac. I'm not sure how to get ringtones into iTunes on a Windows machine.)
Send it to iTunes so you you can sync it over to your iPhone.
Create a new contact called "IGNORE" or something that will let you know it's one of those unwanted callers. Assign the silent ringtone to that contact. (this allows you to keep your default ringtone audible)
As other numbers call that you don't want to be bothered by, you can just add them to that existing contact.
You hear the ringtones from people you want, and you don't from the annoying callers.
Problem solved. :)