Remove Duplicate Messages
1.3.7Removes duplicate messages from the current view in Mail.
Overview
Remove Duplicate Messages removes duplicate messages from the current selection in Apple Mail. The script determines whether a message is a duplicate of another message by examining these message headers:
- MessageID
- Subject
- From
- Date
If these message headers match another message in the view, the script flags the message as a duplicate and removes it from Mail.
The script displays interactive dialog boxes that allow you to specify whether you want duplicate messages to be removed or retained, optionally prompting you for each duplicate message it encounters while scanning. You can also instruct the script to save all duplicate messages to an archive folder on disk before they are removed.
This script is provided with source code so that you may edit the script as you see fit. Read the included instructions for more information.
What's new in Remove Duplicate Messages
- Addressed an issue where deleted messages were not actually deleted from some servers, and subsequently reappeared as a result. Remove Duplicate Messages was changed to move duplicate messages to the Trash mailbox, which causes them to be moved to the Trash mailbox on the server as well. Thanks to Thomas W. Lynch for suggesting this.
- Fixed a bug where certain message archive filenames would exceed 255 characters in length, causing a runtime error.
Remove Duplicate Messages Screenshots
- Intel 64
- Intel 32
- Mac OS X 10.6.8 or later
Install with MacUpdateDownloaded & Installed 6,343 times
10 Remove Duplicate Messages Reviews
Rate this app:
Most helpful
Can someone share the latest version (1.3.6) with me ?
I would appreciate a lot !
TIA
Just tested it and it works very very well.
I used it on local folders where I had a disastrous copy/paste that duplicated thousand of messages.
It browsed more that 12 000 messages and removed 5000 duplicates !
Thank you so much, great script !
-Ben
I've been using your mail script for years now with great success. I've just gone to do another cleanup of duplicates (El Capitain, iCloud account) and while it finds and removes duplicates, they just download again as soon as the script finishes running (so they end up again in the inbox and I can see the ones the script removed in the trash).
Is this expected? I'm using 1.3.5.
Thanks, Tony.
Locally removed duplicates just download again ... too bad.
The script simply tells Mail to delete the message (tell app "Mail" to delete thisMessage). It is then left to Mail to decide how to delete them. Typically they are moved to a Trash folder. If they then come back when you check for new messages, then either Mail isn't telling the server to delete them, or the server is ignoring Mail's commands.
I'm curious: What email provider hosts these problematic messages?
I accidentally left some debugging functions in the 1.3.0 release and didn't notice it because I have that application installed. I just submitted an update to MacUpdate for version 1.3.1 with the fix. You can download it directly here:
http://jollyroger.kicks-ass.org/software/RemoveDuplicateMessages-Mail.zip
Vanilla installs of Yosemite don't appear to have this property yet so you have to create it the first time with the following statement so that the script works without error: "defaults write com.apple.mail _AlwaysShowDuplicates -bool true".
Ideally, there could be an existence check built into the script and/or a statement like the one above to create and set the property before trying to only set it.
After that was fixed up, I used a smart folder to put all of my emails into one "folder" and the script blew through all 16,000 emails in no time at all.
Pretty impressive!
Case in point: I ran the Remove Duplicates script on a folder which contains 3,301 messages. The script found 201 duplicates, and it removed them to the archive folder. I sent those duplicates to the trash. I rebuilt the source mailbox. The source mailbox again had 3,301 messages.
So, I ran the script again on the same source folder. It again found 201 duplicates, and it removed them. The number of messages did not decrement, as I would have expected it to do. I still have the same number of messages in the source folder as when I started.
I ran the script a third time. This time, I selected the option to review each duplicated message before choosing to leave it in the source folder or send it to the archive folder. It again found those same 201 duplicate messages; I again sent all of the duplicate messages to the archive folder; the script again failed to remove those duplicate messages from the source folder in Apple Mail.
I reported this to the developer, but he did not reply.
My low rating is based on the fact that the script does not perform as intended. It finds duplicates, but it does not remove them.
Whether this is a failure of the script or a bug in Apple Mail is a moot issue. A previous Apple Script, developed by Andreas Amann, worked brilliantly before Mac OS X 10.7 (Lion) was developed. But, on his website and through correspondence with him, he points out that Apple introduced a bug in Apple Mail which prevents an Apple Script from running correctly in Apple Mail. He filed bug reports with Apple, but those bugs have not been fixed. Accordingly, he had to abandon further development of his very useful scripts. If you try to run his script on Mavericks, for instance, you will encounter an error that prevents it from getting started. While Jolly Roger's script can get started and it can apparently find duplicate messages, my tests inform me that it can not successfully finish the job of removing those messages, which renders it ineffective.
Since then, I booted up in Yosemite, and I created Mail accounts there.
Then I installed and tested Jolly Roger's Apple Script under Yosemite. I am pleased to report that, on Yosemite, the script worked as designed. It not only located all the duplicate messages, but it also removed them to an archive folder AND it removed them from the source folder in Mail, which it was NOT doing under Mavericks.
In private correspondence with Jolly Roger, he says no else has reported encountering this issue on Mavericks. And, since he no longer uses Mavericks, he cannot trouble-shoot this issue with me. So, I am on my own to see what mysterious issue may be preventing his script from working on my configuration of Mail under Mavericks.
It will be a low priority task, but I will eventually conduct some tests to see what the problem is. It does not appear to lie with his script, though. So, I would recommend his script; the bugs may lie within Apple Mail itself and/or the Mac OS, both of which are out of his control.
In brief, what he has accomplished is commendable. Use the script with confidence.
While at first processing is very fast, as time goes on during a scan the script will naturally slow down more and more. If you watch the log carefully during the scan, you will be able to perceive the slow down as time progresses as you have noted.
The reason the script slows down as time goes on is it must build an internal list of all the messages it has previously encountered so that it can compare each message to that list as it encounters new messages to determine whether they are duplicates or not. I do use some tricks to minimize the amount of time it takes to scan that list, such as only storing the bare minimum amount of information needed about each message, and using AppleScript references when accessing the list, which is far faster than direct list access; but the script is limited by the raw performance of AppleScript and your machine in this regard. It just simply takes time to process a lot of messages.
It doesn't seem to let you decide what to remove. I'd rather have that Option than letting a Script make that decision.
The included "ReadMe" document shows you how to install and use the script. Basically, you just copy the script to your hard drive, select some messages in Apple Mail, and then run the script.
I just posted a new version (1.2) which should be available shortly that allows you to interactively choose which duplicate messages to remove. Thank you for your feedback, and please enjoy! :)
I'm one of those guys who's pretty Tech Savvy in some ways and totally at a loss, in others. I've done a lot in the Terminal, but I'm not too crazy about doing it. So, the more easy it is, the better.
It sounds like a GREAT "Client..." As when I went from Snow Leopard to Mavericks, my Apple Mail was serious screwed-up. The weird thing is that I can't find my old eMail Folders but if I search in Mail, I can find stuff that is not in the Folders I originally created.
http://www.macworld.com/article/2046576/how-to-delete-duplicate-email-messages.html
It's an open-source AppleScript, which you can examine yourself in Script Editor if you wish. There is no malware or any other nefarious code in it. It simply removes duplicate messages from Mail, as stated.
You can download it directly from here:
http://jollyroger.kicks-ass.org/software/RemoveDuplicateMessages-Mail.zip
I assume it's way-too much work to set this up as an Ap? I'd just like to have the choice to pick-and-choose what I keep and what I delete. Thanx!