Is there any harm in using the iPad charger for iPhone 4?
The iPhone simply won't draw as much through the charger as the iPad would. Certainly shouldn't cause a problem at all (and my money's where my mouth is because we just have the iPad charger plugged in these days, using it for our iPad as well as our 2 iPhone 4s - we've been doing this for about 5-6 weeks now without any obvious ill-effects).
No harm; you can use it. Straight from Apple (although last updated Oct 2011, so maybe not there when the question was originally asked and answered):
The iPad USB Power Adapter is a 10W charger. While designed for use with the iPad, you can use the iPad 10W USB Power Adapter to charge all iPhone and iPod models by connecting them using a Dock Connector to USB cable.
The rule of thumb on interchanging charging units is that the rated wattage for a given charger is considered the maximum that the charger can safely deliver, and not what it will deliver all the time.
You can have a thousand watt charger and so long as your phone needs less that that it all that it will supply it happily.
Where you can get unstuck is doing it the other way around. If you charge your iPad over an iPhone charger, it will max out the circuitry as the iPad will want to draw more than the charger can safely do. The charger should remain safe, but it will be operating at or close to maximum capacity, and this may shorten it's life, although so long as you are using an proper Apple one and not an eBay special the protection circuitry against overheating/overvoltage etc should stop it from burning your house down.
I prefer the 10 watt charger that comes with the iPad for my iPhone (4 and 4S) because it charges them faster than the small iPhone charger. I've been charging my iPhones with the iPad charger since getting my first iPad (1) a year and a half ago and not only is no harm done to the phone, it charges many times faster. I bought an extra 10 watt Apple charger that travels with me and bought one for my wife to use on her iPhone in her office at school. All work perfectly and quickly on both iPhone and iPad.