Finding duplicate files without losing the one you need

Two files with the same name are not necessarily the same file, and two files with different names may be identical. Any tool that finds duplicates by name will miss some and wrongly flag others, so it is worth knowing how a careful comparison works.

What counts as a duplicate

A reliable check has three stages. First, compare file size: files of different sizes cannot be identical. Second, compare a content fingerprint, a hash, of the files that share a size. Third, if it matters, compare the files byte by byte. You can do the same by hand for two files: shasum -a 256 one.pdf two.pdf prints a fingerprint for each, and cmp one.pdf two.pdf prints nothing when the contents match.

Name, date and location are useful for choosing which copy to keep, but not for deciding whether two files are copies.

Copies that may not use extra space

On a modern Mac with APFS, some duplicates share their data on disk:

• Clones. When Finder duplicates a file on the same volume, APFS can create a clone that shares the original’s data until one of them changes. Deleting one of two clones frees little or nothing.
• Hard links. Two names can point to the same file. ls -li shows a shared inode number. Removing one name does not free space while another remains.

This is why a list of duplicates and the space you get back are different numbers. A cleaner should count shared data once. This app is being built to report allocated space this way, but the pre-alpha does not yet remove anything, so treat any estimate as a lower bound rather than a promise.

Choosing the copy to keep

Some questions help more than the file’s date:

1. Where does each copy live? A copy in Documents or a project folder is usually the one you meant to keep. A copy in Downloads is usually a leftover.
2. Does something depend on it? A project, a playlist or a photo library may refer to one path. Removing that copy breaks the reference.
3. Is it inside an app’s library? Do not deduplicate inside the Photos or Music library packages. Those apps manage their own files.
4. Is one copy only in the cloud? A placeholder file that is stored only in iCloud is not a second local copy, and removing it can remove it from iCloud.

If you cannot decide, keep both. The cost of an unneeded duplicate is some space. The cost of the wrong deletion can be your only copy.

Removing duplicates safely

Move the extra copy to the Trash rather than deleting it, and keep it there for a while. The Trash guide explains why space returns only when the Trash is emptied.

How the app approaches duplicates

The app is in pre-alpha. The Duplicates section is designed to group matching files, show where each one lives, and let you choose which to keep, with the choice always confirmed before anything moves. The Only safe preselection level does not preselect duplicates, while Balanced does. The scanner today is read-only and cleanup is not built yet.