I had a customer that had a massive music collection with a tremendous amount of duplicates.
I resolved 99% of the duplicates using a multiple program process.
The tools were: iTunes, Media Monkey with a custom script some guy made, and DoubleKiller.
I also want to add Duplicate File Finder to the list of software I've found useful.
So iTunes...
iTunes has an option that lets you see duplicates. You can also hold the SHIFT key as you click the menu and it changes the option to something more specific with duplicates. I don't have iTunes installed so I can't tell you anything more than that, just that iTunes has a method to find duplicates. This is probably the easiest solution for a SMALL library.
The next in the list was
DoubleKiller (
http://www.bigbangenterprises.de/en/doublekiller/). Straight forward program. Finds duplicate files and lets you delete them.
Duplicate File Finder (
http://www.ashisoft.com/) may be in the same category as DoubleKiller. I may have stumbled across it later, I don't remember. I have a lot of tools so I may be doubled up on some. I like DFF because the latest version seems completely free for basic use. I was using it the other day to weed out duplicate picture files.
Finally, we bring out the big guns...
Media Monkey (
http://www.mediamonkey.com/) is a media playback program ala Winamp. The power of it is that you can create script based programs.
The script I used is
AdvancedDuplicateFindFix.3.7.4.mmip (
http://www.mediamonkey.com/forum/viewtopic.php?f=2&t=13000&sid=bb450d88c5b8d1316537ead66e1fe6e3 -- it's at the bottom of the first post; the first post has the instructions). This really nuked a lot of the duplicates because majority of them were iTunes converting an existing file into mp4 format and so this changes the size/crc/name/etc. I believe the power of this script is that it can check ID tags and such.
Hope this helps!
At the very least.. these programs should be in every technicians tools folder. They may only be used once in a while, but they're surely very handy to have.