Code & automationCodeClaudejungtaejoon

dupes.py: find duplicate files in any folder (read-only, nothing to install)

Downloads and photo folders fill up with the same file saved three times (IMG_0412.jpg, IMG_0412 (1).jpg, a copy in another folder). This one-file Python script lists every group of identical files and tells you how much space you would get back. It never deletes anything: you read the list and decide.

Run it

python3 dupes.py ~/Downloads
python3 dupes.py ~/Pictures --min-size 1M --csv dupes.csv

What is good
- Standard library only, works on macOS, Linux and Windows with Python 3.8+.
- Fast on big folders: it compares sizes first, then only the first 64 KB, and reads whole files only when they still match.
- Hard links and symlinks are not counted as copies, hidden folders like .git are skipped unless you pass --hidden.
- Groups are sorted by wasted space, so the biggest wins come first. Copies inside a group are listed oldest first.

What is weak
- It finds byte-identical files only. The same photo resized or re-exported will not show up.
- No "delete for me" option, on purpose. With thousands of groups, going through the list by hand is tedious (the CSV helps: open it in a spreadsheet).
- Files that change while it runs, or files on cloud drives that are not downloaded yet, can be skipped or slow.

Tested on a scratch folder with copies, a hard link, a hidden copy and two files that share their first 70 KB but differ at the end (correctly reported as different).

Would you keep this in your toolbox, or is it slop next to the cleanup apps you already have?

Prompt used
Write a single-file Python 3 script (standard library only) that finds duplicate files under a folder. It must be read-only: never delete or move anything. Group by size, then by a hash of the first 64 KB, then by a full hash. Skip symlinks and hard links to a file already seen, skip hidden folders unless --hidden, support --min-size (like 100k, 1M) and --csv output. Sort groups by wasted space and print how much space could be freed.

#python #script #files #cleanup

c/code · made with Claude · ▲ 0 gem · ▼ 0 slop · 0 comments