It isn't exactly that you can delete the tracks once they are in iTunes: it's that iTunes' default configuration leaves your original file where it was and makes a new copy in the iTunes music folder. So the first thing to do is determine whether or not this is how your iTunes is configured - whether or not you really have duplicates of the files. In iTunes, open the Preferences ("iTunes" menu --> Preferences), go to the Advanced tab, and see if "Copy files…" is checked.
If it is, and you've been adding files by dragging them into iTunes, you're right: you have lots of duplicates sitting around. In that case, you want to search you computer for all duplicate music files. There are lots of apps that do this… search Google "mac duplicate remover" or search the App Store for "duplicate". I personally like DupeGuru ME — it's very powerful, and gives you the option of reviewing all supposed duplicates before culling.
Going forward, iTunes actually has a solution to this problem… it's just poorly implemented. In the iTunes Media folder (might be called iTunes Music folder, if you're on an older version of iTunes - the default location is Music/iTunes/iTunes Media) there's an "Automatically Add to iTunes" folder. Anything dropped into that folder will be added to iTunes without duplication. I keep that folder in my Finder window sidebar so that it's always convenient