Remove Duplicate Messages
Remove Duplicate Messages
1.3.7

3.7

Remove Duplicate Messages free download for Mac

Remove Duplicate Messages

1.3.7
14 August 2019

Removes duplicate messages from the current view in Mail.

Overview

Remove Duplicate Messages removes duplicate messages from the current selection in Apple Mail. The script determines whether a message is a duplicate of another message by examining these message headers:

  • MessageID
  • Subject
  • From
  • Date

If these message headers match another message in the view, the script flags the message as a duplicate and removes it from Mail.

The script displays interactive dialog boxes that allow you to specify whether you want duplicate messages to be removed or retained, optionally prompting you for each duplicate message it encounters while scanning. You can also instruct the script to save all duplicate messages to an archive folder on disk before they are removed.

This script is provided with source code so that you may edit the script as you see fit. Read the included instructions for more information.

What's new in Remove Duplicate Messages

Version 1.3.7:
  • Addressed an issue where deleted messages were not actually deleted from some servers, and subsequently reappeared as a result. Remove Duplicate Messages was changed to move duplicate messages to the Trash mailbox, which causes them to be moved to the Trash mailbox on the server as well. Thanks to Thomas W. Lynch for suggesting this.
  • Fixed a bug where certain message archive filenames would exceed 255 characters in length, causing a runtime error.

Join over 500,000 subscribers.

Subscribe for our newsletter with best Mac offers from MacUpdate.

10 Remove Duplicate Messages Reviews

Rate this app:

anthonyproberts
23 March 2015

Most helpful

Hey Jolly, Great script. I'm using 1.2.2 on an SSD i7 but find that it slows down from processing about 10 messages a second to about 2 a second after about 3000-4000 messages processed. Is this just applescript or is something not right with 1.2.2? Tony
Like (1)
Version 1.2.2
opnsp
15 November 2017
Seems promising but unfortunately the download link is dead...
Can someone share the latest version (1.3.6) with me ?
I would appreciate a lot !


TIA
Like
Version 1.3.6
3 answer(s)
Ribero
Ribero
17 November 2017
Link available again.
Just tested it and it works very very well.
I used it on local folders where I had a disastrous copy/paste that duplicated thousand of messages.

It browsed more that 12 000 messages and removed 5000 duplicates !

Thank you so much, great script !
Like
sjbyoo
sjbyoo
25 February 2018
The Link //jollyroger.kicks-ass.org/software/ is asking me for my username and password. I used the ones from macupdate.com but it says '401 Unauthorized'. Can someone help me? I used this software a few years ago on my MacBookPro and worked well. I got a new Mac and I need to use it again. Thanks!
-Ben
Like
peterr-2
peterr-2
29 October 2018
I am currently running this script and it actually does find some duplicates ... but the files that are moved to or created in the "Archived Duplicates" folder are all 3k to 28k in size ... what about attachments, will they still fill up my mailbox or is there a way to remove them too ?
Like
Christian-Bauer
09 March 2017
Works exact, but is rather slow as it has to communicate with Mail a lot. It looks up 1000 mails per minute and removes about 50 per minute.
Like
Version 1.3.5
anthonyproberts
18 June 2016
Hello Jolly,

I've been using your mail script for years now with great success. I've just gone to do another cleanup of duplicates (El Capitain, iCloud account) and while it finds and removes duplicates, they just download again as soon as the script finishes running (so they end up again in the inbox and I can see the ones the script removed in the trash).

Is this expected? I'm using 1.3.5.

Thanks, Tony.
Like
Version 1.3.5
2 answer(s)
un-it
un-it
15 August 2016
I have the very same problem (with v1.3.5).
Locally removed duplicates just download again ... too bad.
Like
JollyRoger8X
JollyRoger8X
03 July 2017
Sorry to hear that you are having this problem.

The script simply tells Mail to delete the message (tell app "Mail" to delete thisMessage). It is then left to Mail to decide how to delete them. Typically they are moved to a Trash folder. If they then come back when you check for new messages, then either Mail isn't telling the server to delete them, or the server is ignoring Mail's commands.

I'm curious: What email provider hosts these problematic messages?
Like
Oo0-bailey-0oO
27 August 2015
With 1.3.0 and OS X 10.10.5, I get a few errors at the hash initialization step: - "Where is ASObjC Runner.app?" pop-up from Script Editor - "2015-08-27 18:25:51.865 defaults[5049:60610] The domain/default pair of (/Users/[username]/Library/Containers/com.apple.mail/Data/Library/Preferences/com.apple.mail, _AlwaysShowDuplicates) does not exist" error pop-up I'm not sure how to proceed from there though. The terminal window never moves on from the first pop-up even if you cancel it. Hopefully mine is a unique case. I'm not sure what's different about my set up though.
Like (1)
Version 1.3.0
4 answer(s)
JollyRoger8X
JollyRoger8X
28 August 2015
Thanks very much for the report! I apologize for the inconvenience.

I accidentally left some debugging functions in the 1.3.0 release and didn't notice it because I have that application installed. I just submitted an update to MacUpdate for version 1.3.1 with the fix. You can download it directly here:

http://jollyroger.kicks-ass.org/software/RemoveDuplicateMessages-Mail.zip
Like (2)
Oo0-bailey-0oO
Oo0-bailey-0oO
28 August 2015
That's great, thanks!
Like (1)
Oo0-bailey-0oO
Oo0-bailey-0oO
29 August 2015
So for the second error: "The domain/default pair of (/Users/[username]/Library/Containers/com.apple.mail/Data/Library/Preferences/com.apple.mail, _AlwaysShowDuplicates) does not exist"
Vanilla installs of Yosemite don't appear to have this property yet so you have to create it the first time with the following statement so that the script works without error: "defaults write com.apple.mail _AlwaysShowDuplicates -bool true".
Ideally, there could be an existence check built into the script and/or a statement like the one above to create and set the property before trying to only set it.

After that was fixed up, I used a smart folder to put all of my emails into one "folder" and the script blew through all 16,000 emails in no time at all.

Pretty impressive!
Like (1)
JollyRoger8X
JollyRoger8X
29 August 2015
Crap! In my haste to get a fix out, I forgot to address that issue in the update. This latest version, 1.3.2, (posting to MacUpdate now) includes a fix for that issue. Thanks so much for your patience!
Like (2)
bowlerboy-jmb
07 July 2015
Unfortunately, this script does not work properly for me. It finds duplicate messages OK, but it does not also remove those duplicate messages from the selected Apple Mail folder, which makes the script useless.

Case in point: I ran the Remove Duplicates script on a folder which contains 3,301 messages. The script found 201 duplicates, and it removed them to the archive folder. I sent those duplicates to the trash. I rebuilt the source mailbox. The source mailbox again had 3,301 messages.

So, I ran the script again on the same source folder. It again found 201 duplicates, and it removed them. The number of messages did not decrement, as I would have expected it to do. I still have the same number of messages in the source folder as when I started.

I ran the script a third time. This time, I selected the option to review each duplicated message before choosing to leave it in the source folder or send it to the archive folder. It again found those same 201 duplicate messages; I again sent all of the duplicate messages to the archive folder; the script again failed to remove those duplicate messages from the source folder in Apple Mail.

I reported this to the developer, but he did not reply.

My low rating is based on the fact that the script does not perform as intended. It finds duplicates, but it does not remove them.

Whether this is a failure of the script or a bug in Apple Mail is a moot issue. A previous Apple Script, developed by Andreas Amann, worked brilliantly before Mac OS X 10.7 (Lion) was developed. But, on his website and through correspondence with him, he points out that Apple introduced a bug in Apple Mail which prevents an Apple Script from running correctly in Apple Mail. He filed bug reports with Apple, but those bugs have not been fixed. Accordingly, he had to abandon further development of his very useful scripts. If you try to run his script on Mavericks, for instance, you will encounter an error that prevents it from getting started. While Jolly Roger's script can get started and it can apparently find duplicate messages, my tests inform me that it can not successfully finish the job of removing those messages, which renders it ineffective.
Like
Version 1.2.3
1 answer(s)
bowlerboy-jmb
bowlerboy-jmb
23 July 2015
Let me amend and append my original report, which, I remind you, was done when I was booted up in Mavericks on my three-partition start-up hard drive.

Since then, I booted up in Yosemite, and I created Mail accounts there.

Then I installed and tested Jolly Roger's Apple Script under Yosemite. I am pleased to report that, on Yosemite, the script worked as designed. It not only located all the duplicate messages, but it also removed them to an archive folder AND it removed them from the source folder in Mail, which it was NOT doing under Mavericks.

In private correspondence with Jolly Roger, he says no else has reported encountering this issue on Mavericks. And, since he no longer uses Mavericks, he cannot trouble-shoot this issue with me. So, I am on my own to see what mysterious issue may be preventing his script from working on my configuration of Mail under Mavericks.

It will be a low priority task, but I will eventually conduct some tests to see what the problem is. It does not appear to lie with his script, though. So, I would recommend his script; the bugs may lie within Apple Mail itself and/or the Mac OS, both of which are out of his control.

In brief, what he has accomplished is commendable. Use the script with confidence.
Like (1)
Oo0-bailey-0oO
03 July 2015
I'm grateful that someone took the time to create such a useful script. Duplicate emails can accumulate from time to time, whether through faulty software (MailHub for instance) or numerous migrations over the years. A couple of issues about the current version: 1) For Yosemite, the "show duplicates" property has changed from "defaults write com.apple.mail AlwaysShowDuplicates -bool true" to "defaults write com.apple.mail _AlwaysShowDuplicates -bool true". Very subtle difference 2) The script times out after about 8,000 messages in my case. I'll leave it for a day and see that no progress has been made, but if I cancel and restart the script, it'll advance by 500 messages (i.e. makes it to 8,500) before hanging again. I suspect the disabling of "include when getting new mail" property is not working as it should, but I checked the AppleScript library and the statement looks fine so I'm not sure. Perhaps a brute force *detect time-out* + *cancel and restart script* is possible if the root cause can't be found. Thanks again for your efforts!
Like (1)
Version 1.2.3
anthonyproberts
23 March 2015
Hey Jolly, Great script. I'm using 1.2.2 on an SSD i7 but find that it slows down from processing about 10 messages a second to about 2 a second after about 3000-4000 messages processed. Is this just applescript or is something not right with 1.2.2? Tony
Like (1)
Version 1.2.2
3 answer(s)
JollyRoger8X
JollyRoger8X
23 March 2015
Hi Tony,

While at first processing is very fast, as time goes on during a scan the script will naturally slow down more and more. If you watch the log carefully during the scan, you will be able to perceive the slow down as time progresses as you have noted.

The reason the script slows down as time goes on is it must build an internal list of all the messages it has previously encountered so that it can compare each message to that list as it encounters new messages to determine whether they are duplicates or not. I do use some tricks to minimize the amount of time it takes to scan that list, such as only storing the bare minimum amount of information needed about each message, and using AppleScript references when accessing the list, which is far faster than direct list access; but the script is limited by the raw performance of AppleScript and your machine in this regard. It just simply takes time to process a lot of messages.
Like
anthonyproberts
anthonyproberts
23 March 2015
Thanks Jolly, I thought as much :)
Like (1)
JollyRoger8X
JollyRoger8X
28 August 2015
Note that the latest version (1.3.1) is significantly faster, especially with large numbers of messages.
Like
Albabe
24 February 2015
I'd love to try this Client but its a .scptd file and I am unfamiliar with those.

It doesn't seem to let you decide what to remove. I'd rather have that Option than letting a Script make that decision.
Like
Version 1.1.2
2 answer(s)
Jolly Roger
Jolly Roger
25 February 2015
Hi Albabe. Thank you for commenting.

The included "ReadMe" document shows you how to install and use the script. Basically, you just copy the script to your hard drive, select some messages in Apple Mail, and then run the script.

I just posted a new version (1.2) which should be available shortly that allows you to interactively choose which duplicate messages to remove. Thank you for your feedback, and please enjoy! :)
Like
Albabe
Albabe
26 February 2015
Jolly Guy!

I'm one of those guys who's pretty Tech Savvy in some ways and totally at a loss, in others. I've done a lot in the Terminal, but I'm not too crazy about doing it. So, the more easy it is, the better.

It sounds like a GREAT "Client..." As when I went from Snow Leopard to Mavericks, my Apple Mail was serious screwed-up. The weird thing is that I can't find my old eMail Folders but if I search in Mail, I can find stuff that is not in the Folders I originally created.
Like
Thinking-Differently
28 January 2015
It seems that while the program may support 10.6.8, MacUpdate no longer does. If I attempt to download it, I get a new window that says "Leave MacUpdate and download..." However, that doesn't work on 10.6.8. So, when you try to go to the site, you discover it is named "Jolly Roger" and is showing the skull and bones of a pirate flag. It makes one nervous about using the software, if they could get it. If this is real, it would be a really useful piece of software for which I would be extremely grateful to the developer for. I hope it is real, works and has no malware. But this all makes one a bit uneazy.
Like
Version 1.1
3 answer(s)
JollyRoger8X
JollyRoger8X
31 January 2015
It's real and was featured in MacWorld:

http://www.macworld.com/article/2046576/how-to-delete-duplicate-email-messages.html

It's an open-source AppleScript, which you can examine yourself in Script Editor if you wish. There is no malware or any other nefarious code in it. It simply removes duplicate messages from Mail, as stated.

You can download it directly from here:

http://jollyroger.kicks-ass.org/software/RemoveDuplicateMessages-Mail.zip
Like
Albabe
Albabe
25 February 2015
Hey Jolly!

I assume it's way-too much work to set this up as an Ap? I'd just like to have the choice to pick-and-choose what I keep and what I delete. Thanx!
Like
Thinking-Differently
Thinking-Differently
26 February 2015
Thanks Jolly Roger. I am now able to get your software. And thanks for still supported Snow Leopard. Many of us think that it is the most stable OS Apple has produced in a while.
Like (1)
Foulger
26 January 2015
Authors web site just shows a blank web site
Like
Version 1.1
1 answer(s)
JollyRoger8X
JollyRoger8X
31 January 2015
http://jollyroger.kicks-ass.org/software/
Like