0

I have a folder with many book files in different formats e.g. .txt, .epub, .rtf, etc.

Many of them are duplicates but have different names e.g.:

Brooks, Terry - Shannara 1 - The Sword of Shannara.txt

Brooks, Terry - Shannara 01 - The Sword of Shannara.txt

If I check these files with filemerge they are exactly the same file.

I have tried running several programs like Gemini on the folder in question with no results.

Every suggestion online seems to involve comparing two folders. I do NOT want to compare two folders. I only want to find the duplicate files within a directory or its sub-directories.

I was looking for a script or terminal command to loop through the folder(s) and show which files have the same content even though they may have different filenames, creation dates, modification dates, etc.

Any help would be much appreciated.

[edit] - I should clarify that I want to find a way to compare two files even though they may be named differently but are of the same type. e.g. text files, or epub files. I do not wish to compare them between types a text file and an epub, of couse, would be different.

Improve this question
1
  • 3
    book.txt & book.epub etc are not going to have the same format or file contents, even if the contained text looks identical when you read the formatted result. I think you're going to have to sort by hand.
    – Tetsujin
    Jul 16, 2021 at 13:13

1 Answer 1

Reset to default
1

fdupes may be your solution. It can be installed via homebrew with

brew install fdupes

If you don’t have homebrew, you can install homebrew with

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Running fdupes -r <directory> will list duplicate files within the directory, even if they’re not in the same folder. fdupes compares based on file contents, not names, so the names being different will not be a problem.

You can use the -d option to be prompted which files to keep (deleting all others), but I would suggest doing the deletions manually.

Improve this answer
7
  • 1
    …but the contents will not be the same.
    – Tetsujin
    Jul 17, 2021 at 7:49
  • However that wil;l not show that ebbok.txt and ebook.epub are the same - and I suspect that if you had 2 epubs for the same book this would not find then as the metadata would differ. Duplicate txtx or possibly html rtf would work
    – mmmmmm
    Jul 17, 2021 at 7:52
  • In the example, both files are .txt. I took that to mean that duplicates are the same format, but different books are in different formats. As far as I know, it’s not possible to compare the text in epub and txt files.
    – tjcaul
    Jul 17, 2021 at 8:08
  • I’m terms of epub metadata: I’ll leave it to someone else to write a program for parsing epub files and removing their metadata.
    – tjcaul
    Jul 17, 2021 at 8:11
  • 1
    You could use Calibre to convert them all to plaintext, or even to sort them; once by author, once by title… but that would be more trouble than just dropping them all in one folder & going through it by hand
    – Tetsujin
    Jul 17, 2021 at 9:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .