image-duplicate-remover

Remove duplicate images from the two given directories recursively

Too many images that are the same but might have different dates and other metadata. This tool compares images based on the following criteria:

• [x] SHA-256 hash of the file, since v0.1.0
• [x] File size in bytes, will be in v0.2.0
• [ ] Width and height
• [ ] Bit dept
• [ ] Number of unique colors
• [ ] Image contents comparison

The idea is to start from stronger method, while progressing towards a weaker method.

Please note that the minimum supported version of Node.js is 18.12.0, which is the active Long Term Support (LTS) version.

Underneath SQLite is used for storing any meta information about the files, as sometimes the amount of files to compare is huge.

It will be possible to reuse the resulting database file and skip reading the files, in the v0.2.0 release. Reading the information from image file is the most time consuming part within the execution of this tool.

Installation

Starting with version 0.3.0, additional tools are needed.

Make sure to have GraphicsMagick (minimum version 1.3.18) available in the PATH. It can be installed for example in Mac via Brew:

brew install graphicsmagick

In Ubuntu it is usually available via:

sudo apt-get install graphicsmagick

Windows users could download an installer package.

Now install the command line tool globally, which might need increased privileges:

[sudo] npm install --global image-duplicate-remover

Please note that while in Linux and with sudo, some of the dependencies might fail to install, which can be fixed in some case by sudo npm install --global --unsafe-perm image-duplicate-remover. See more details at docs.npmjs.com.

Command line options

The available command line options can be seen via command:

image-duplicate-remover --help

The output will be something similar to:

image-duplicate-remover [options] <primary directory> <secondary directory>

  -h, --help             Help and usage instructions
  -V, --version          Version number
  -v, --verbose          Verbose output, will print which file is currently being processed
  -D, --database String  SQLite database to use
  -S, --skip-reading     Skip reading the directories, just use the existing database. Requires database
  -n, --dry-run          Try it out without actually removing anything

Version 0.4.0

Example commands

The following command shows how two folders are compared, but nothing will be removed since the --dry-run option is used.

image-duplicate-remover -vn a b

Contributing

First thing to do is to file an issue.

"A Beginner's Guide to Open Source: The Best Advice for Making your First Contribution".

Also there is a blog post about "45 Github Issues Dos and Don’ts".

Linting is done with ESLint and can be executed with npm run lint. There should be no errors appearing after any JavaScript file changes.

npm install
npm run lint

Automated testing is done against Ubuntu Trusty at Travis CI and Windows at AppVeyor.

Unit tests are written with tape and can be executed with npm test. Code coverage is inspected with nyc and can be executed with npm run coverage after running npm test. Please make sure it is over 90% at all times.

Version history

Changes happening across different versions and upcoming changes are tracked in the CHANGELOG.md file.

License

Licensed under the MIT license.

Copyright (c) Juga Paazmaya [email protected]