If you have a lot of duplicate files "fdupes" can help you remove them very easily.
On Debian derived systems you can install fdupes using the command:
# apt-get install fdupes
The program is run from the terminal and the options are very basic. To search a directory for duplicate files, simply run the following command:
$ fdupes -r my_dir/
The command will recursively search the directory
my_dir for any duplicate files, and display the results.
You can also create a text file containing the names of you duplicate files running this command:
$ fdupes -r my_dir/ > my_duplicate_files.txt
What I needed was to remove all duplicate files except for one of each file. I also needed a list of the files that was kept and the ones that was removed. To do that I used this command:
$ fdupes -rdN my_dir/ > dubs.txt
r option makes fdupes search for files recursively. The
d option makes fdupes delete duplicates. The
N option, when used together with
d, preserve the first file in each set of duplicates and delete the others without prompting the user.
man fdupes for further instructions.