# Duplicate MP3 file detector program **URL:** \ **Category:** Factual Questions\ **Created:** [March 19, 2003, 7:06am UTC](https://boards.straightdope.com/t/duplicate-mp3-file-detector-program/162494 "2003-03-19T07:06:28Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Harmonix](https://avatars.discourse-cdn.com/v4/letter/h/3ec8ea/32.png) [@Harmonix](https://boards.straightdope.com/u/Harmonix)\ **Post date:** [March 19, 2003, 7:06am UTC](https://boards.straightdope.com/t/duplicate-mp3-file-detector-program/162494/1 "2003-03-19T07:06:28Z")
I’d imagine someone would have programmed one of these by now. I can’t seem to find any decently made. I’d imagine one which did a bit by bit comparison of files with say… a 95+% similarity would be extremely accurate(as well as taking _forever_). So anyone know of any such programs? ---
**Author:** ![coffeecat](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/coffeecat/32/3405_2.png) [@coffeecat](https://boards.straightdope.com/u/coffeecat)\ **Post date:** [March 19, 2003, 7:28am UTC](https://boards.straightdope.com/t/duplicate-mp3-file-detector-program/162494/2 "2003-03-19T07:28:40Z")
[Fdupes](http://netdial.caribe.net/~adrian2/fdupes.html) will do it on Linux–for any file, not just mp3s. ---
**Author:** ![SmackFu](https://avatars.discourse-cdn.com/v4/letter/s/53a042/32.png) [@SmackFu](https://boards.straightdope.com/u/SmackFu)\ **Post date:** [March 19, 2003, 2:42pm UTC](https://boards.straightdope.com/t/duplicate-mp3-file-detector-program/162494/3 "2003-03-19T14:42:05Z")
The problem is that there may be no bit similarity between MP3s of the same song that are at different bitrates or CBR vs. VBR. The same problem comes up with identifying anonymous MP3s. Generally, the only way to do it properly is to decode the first x seconds of the song and convert that into a checksum-like signature. Then you can compare all of the signatures to check for duplicates. (Or look them up in a database to identify the MP3) I can’t find any software that explicitly checks for duplicates in this way. [The MusicBrainz TRM Generator](http://www.musicbrainz.org/products/trmgen/download.html) will create signature files, but it doesn’t use them for anything. ---
**Author:** ![Revtim](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/revtim/32/1042_2.png) [@Revtim](https://boards.straightdope.com/u/Revtim)\ **Post date:** [March 19, 2003, 2:58pm UTC](https://boards.straightdope.com/t/duplicate-mp3-file-detector-program/162494/4 "2003-03-19T14:58:08Z")
Even if they at the same bitrate, if they were made with different encoders they will not be bit-for-bit identical. Th best strategy, I believe, would be for a program to look for similarities in the file name. There was an fdupes program for the PC many years ago, that ran on DOS. IIRC it only found files that were exactly the same with the same file name, though. ---
**Author:** ![EasyPhil](https://avatars.discourse-cdn.com/v4/letter/e/e47c2d/32.png) [@EasyPhil](https://boards.straightdope.com/u/EasyPhil)\ **Post date:** [March 19, 2003, 3:17pm UTC](https://boards.straightdope.com/t/duplicate-mp3-file-detector-program/162494/5 "2003-03-19T15:17:45Z")
What’s wrong with using the search function from Windows for \*.mp3, arranging the remaining list by name and then sorting out the duplicates? ---
**Author:** ![Revtim](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/revtim/32/1042_2.png) [@Revtim](https://boards.straightdope.com/u/Revtim)\ **Post date:** [March 19, 2003, 3:24pm UTC](https://boards.straightdope.com/t/duplicate-mp3-file-detector-program/162494/6 "2003-03-19T15:24:49Z")
Some might have song name first, and other might have artist first, or album, or whatever. ---
**Author:** ![MysteryFellow63427](https://avatars.discourse-cdn.com/v4/letter/m/a88e4f/32.png) [@MysteryFellow63427](https://boards.straightdope.com/u/MysteryFellow63427)\ **Post date:** [March 19, 2003, 5:51pm UTC](https://boards.straightdope.com/t/duplicate-mp3-file-detector-program/162494/7 "2003-03-19T17:51:44Z")
Sounds like a good programming project… :dubious: ---
**Author:** ![Harmonix](https://avatars.discourse-cdn.com/v4/letter/h/3ec8ea/32.png) [@Harmonix](https://boards.straightdope.com/u/Harmonix)\ **Post date:** [March 19, 2003, 7:05pm UTC](https://boards.straightdope.com/t/duplicate-mp3-file-detector-program/162494/8 "2003-03-19T19:05:34Z")
> [@](#): > > \*Originally posted by neuroman \* > \*\*Sounds like a good programming project… :dubious: \*\* Well considering i’m a Biotech major and i’m barely going to take my 2nd programming class for my computer science minor next quarter I don’t think i’m qualified juuuusst yet… 🙂 > [@](#): > > The problem is that there may be no bit similarity between MP3s of the same song that are at different bitrates or CBR vs. VBR. The same problem comes up with identifying anonymous MP3s. Generally, the only way to do it properly is to decode the first x seconds of the song and convert that into a checksum-like signature. Then you can compare all of the signatures to check for duplicates. (Or look them up in a database to identify the MP3) > > I can’t find any software that explicitly checks for duplicates in this way. The MusicBrainz TRM Generator will create signature files, but it doesn’t use them for anything. I was discussing this with a programmer friend of mine. Now I realize how stupid bit by bit comparison would be. But he told me about a program that compared the mp3’s against an online database with approximately 90% accuracy. But alas, he forgot what it was. Question still stands, anyone know of a mp3 duplicate pruning program? ---
**Author:** ![Harmonix](https://avatars.discourse-cdn.com/v4/letter/h/3ec8ea/32.png) [@Harmonix](https://boards.straightdope.com/u/Harmonix)\ **Post date:** [March 19, 2003, 7:14pm UTC](https://boards.straightdope.com/t/duplicate-mp3-file-detector-program/162494/9 "2003-03-19T19:14:59Z")
ok, so I looked up smack fu’s linked program. It seems their method would work nicely to prune mp3’s… but it doesn’t seem to actually do it. anyone know of anything that does? ---
**Author:** ![EasyPhil](https://avatars.discourse-cdn.com/v4/letter/e/e47c2d/32.png) [@EasyPhil](https://boards.straightdope.com/u/EasyPhil)\ **Post date:** [March 19, 2003, 7:59pm UTC](https://boards.straightdope.com/t/duplicate-mp3-file-detector-program/162494/10 "2003-03-19T19:59:23Z")
You can download a program called Media Jukebox, it will scan your hard drive and create a catalog by album, artist, genre etc. I believe it reads the ID3 tags in the file to determine aritist, album and genre. This may be close to what you’re looking for. ---
**Author:** ![Harmonix](https://avatars.discourse-cdn.com/v4/letter/h/3ec8ea/32.png) [@Harmonix](https://boards.straightdope.com/u/Harmonix)\ **Post date:** [March 19, 2003, 9:21pm UTC](https://boards.straightdope.com/t/duplicate-mp3-file-detector-program/162494/11 "2003-03-19T21:21:41Z")
I’ll look into it, but i’m still lookin’ for a better solution.