# Locate Duplicate files **URL:** \ **Category:** Factual Questions\ **Created:** [December 3, 2007, 10:51am UTC](https://boards.straightdope.com/t/locate-duplicate-files/428644 "2007-12-03T10:51:26Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Richard\_Pearse](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/richard_pearse/32/13144_2.png) [@Richard\_Pearse](https://boards.straightdope.com/u/Richard_Pearse)\ **Post date:** [December 3, 2007, 10:51am UTC](https://boards.straightdope.com/t/locate-duplicate-files/428644/1 "2007-12-03T10:51:26Z")
Is there software available that will search your hard drive and locate duplicate files? My computer file system is a bit of a mess! :smack: ---
**Author:** ![Derleth](https://avatars.discourse-cdn.com/v4/letter/d/b9e5f3/32.png) [@Derleth](https://boards.straightdope.com/u/Derleth)\ **Post date:** [December 3, 2007, 11:49am UTC](https://boards.straightdope.com/t/locate-duplicate-files/428644/2 "2007-12-03T11:49:52Z")
Yes. What OS are you running? ---
**Author:** ![Richard\_Pearse](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/richard_pearse/32/13144_2.png) [@Richard\_Pearse](https://boards.straightdope.com/u/Richard_Pearse)\ **Post date:** [December 3, 2007, 12:51pm UTC](https://boards.straightdope.com/t/locate-duplicate-files/428644/3 "2007-12-03T12:51:07Z")
Sorry. It’s Windows XP with SP2. ---
**Author:** ![Small\_Clanger](https://avatars.discourse-cdn.com/v4/letter/s/9fc348/32.png) [@Small\_Clanger](https://boards.straightdope.com/u/Small_Clanger)\ **Post date:** [December 3, 2007, 1:03pm UTC](https://boards.straightdope.com/t/locate-duplicate-files/428644/4 "2007-12-03T13:03:34Z")
Google _Windows “Space Hound”_ ---
**Author:** ![scr4](https://avatars.discourse-cdn.com/v4/letter/s/59ef9b/32.png) [@scr4](https://boards.straightdope.com/u/scr4)\ **Post date:** [December 3, 2007, 1:47pm UTC](https://boards.straightdope.com/t/locate-duplicate-files/428644/5 "2007-12-03T13:47:01Z")
I use [CloneSpy](http://www.clonespy.com/). Powerful and flexible, and free to use (but donations accepted - and I already donated). I particularly like the various ways you can specify which duplicate to delete (e.g. if duplicate found, delete the one with the longer filename, or delete the one in this folder, etc). ---
**Author:** ![Uncommon\_Sense](https://avatars.discourse-cdn.com/v4/letter/u/f9ae1b/32.png) [@Uncommon\_Sense](https://boards.straightdope.com/u/Uncommon_Sense)\ **Post date:** [December 3, 2007, 5:13pm UTC](https://boards.straightdope.com/t/locate-duplicate-files/428644/6 "2007-12-03T17:13:26Z")
This is good news, I did not realize such a thing existed. I will be using it whence I get home. Can these things detect duplicate files on an entire home network? Cause that would be the greatest thing ever. ---
**Author:** ![danceswithcats](https://avatars.discourse-cdn.com/v4/letter/d/9e8a1a/32.png) [@danceswithcats](https://boards.straightdope.com/u/danceswithcats)\ **Post date:** [December 3, 2007, 5:55pm UTC](https://boards.straightdope.com/t/locate-duplicate-files/428644/7 "2007-12-03T17:55:54Z")
I have one called Duplicate File Finder from [www.funduc.com](http://www.funduc.com) which is freeware. It worked nicely. ---
**Author:** ![butler1850](https://avatars.discourse-cdn.com/v4/letter/b/779978/32.png) [@butler1850](https://boards.straightdope.com/u/butler1850)\ **Post date:** [December 3, 2007, 9:11pm UTC](https://boards.straightdope.com/t/locate-duplicate-files/428644/8 "2007-12-03T21:11:36Z")
Has anyone used any of these on large corporate type shares? 500Gb volumes? Across multiple volumes? ---
**Author:** ![Stan\_Doubt](https://avatars.discourse-cdn.com/v4/letter/s/53a042/32.png) [@Stan\_Doubt](https://boards.straightdope.com/u/Stan_Doubt)\ **Post date:** [December 3, 2007, 11:17pm UTC](https://boards.straightdope.com/t/locate-duplicate-files/428644/9 "2007-12-03T23:17:49Z")
Finding the dupes is the easy part. Finding a way to safely and logically delete the duplicates is the hard part. On my own Windows machines I use [Double Killer](http://www.bigbangenterprises.de/en/doublekiller/) for most file types because it can automate the process of deleting one or more of the dupes. If I screw something up it is my own fault. For images, I use [Dup Detector](http://www.keronsoft.com/dupdetector.html) because it can compare image files pixel to pixel to find duplicates regardless of file type or size. On the corporate network where I serve as the server bitch, we were faced with a storage crisis before getting our new server. I couldn’t make any assumptions about which version(s) of a particular file to delete. So I used [Treesize Professional](http://www.jam-software.com/treesize/), a very user-friendly means of visualizing (and copying) the structure of files on a drive. While it would’ve been possible to use a command line solution like dir to just get the file listing, I needed to create a safe, nice and easy GUI based workflow that an intern could run to find large duplicated files on the corporate server (~1.5million files, ~250GB). The output from Treesize was dumped into an Access database that ran a query and generated a report, from which he contacted the owner of the largest files to determine the correct course of action. We didn’t even bother with the small potatoes, unless it was an entire duplicated directory. I think Double Killer or Double Killer Pro could be set up to do the same thing, but again, it is really not the finding that is the problem, it is the fact that there is usually no logical means of determining which file is safe to delete, and this goes **double** when you are dealing with other people’s files. ---
**Author:** ![butler1850](https://avatars.discourse-cdn.com/v4/letter/b/779978/32.png) [@butler1850](https://boards.straightdope.com/u/butler1850)\ **Post date:** [December 4, 2007, 5:05pm UTC](https://boards.straightdope.com/t/locate-duplicate-files/428644/10 "2007-12-04T17:05:46Z")
[QUOTE=Stan Doubt] \ The output from Treesize was dumped into an Access database that ran a query and generated a report, from which he contacted the owner of the largest files to determine the correct course of action. We didn’t even bother with the small potatoes, unless it was an entire duplicated directory. [/QUOTE] We started with that, moved to Size Explorer pro, and now have a scripted process to send emails automatically with custom spreadsheets which we can send to all users on the volume, based on minimum total usage. The problem is the number of volumes for us, and an inability to easily scan for actual duplicates. I’ve tried many of the solutions, but the number, and size of the volumes has made scans simply take too long. ☹ (40 or servers, with nearly 80Tb) :eek: Of course, they don’t actually do anything with the files, and so many people have moved through the company (40K+ employees) that it’s often tough to find who really owns the file, or can make a decision. Ah, the joys of cheap disk, and lazy users. -Butler