Large messages causing mail loops
T. Joseph Carter
tjcarter at spiritsubstance.com
Wed Jul 16 14:22:21 BST 2014
I've run into an issue with OfflineIMAP that has caused some MASSIVE
duplication with dreamhost. This is somewhat difficult for me to
debug as my immediate reaction to finding gigabytes upon gigabytes of
duplicated mail is to immediately shut things down and start purging.
(There's also the little problem that this command did not terminate
and was counting down to its next refresh…
tjcarter at amaya:~$ offlineimap -d imap -1 -o -a tjcarter
… because the account in question has
idlefolders = [ 'INBOX' ]
specified. But that's a fairly minor niggle since idle is still
indicated to be somewhat experimental.)
Anyway, how it tends to happen:
I will send a fairly large file via email (significantly > 5 MiB)
from a client that doesn't easily tell you how big the thing you're
sending happens to be. Like a phone, for example.
The email may or may not deliver, depending on what the destination
server does with zomg-huge emails. It may deliver, it may bounce, or
it may be silently deleted. I've seen 35 MiB files send just fine to
some, and emails (after encoding) as small as 1000000 total bytes
silently discarded. Regardless, the client stores its copy in
INBOX/Sent on the server, as expected.
Then offlineimap downloads this massive email. Then it becomes
convinced that this email is new and uploads the email. Then it
downloads the "new" email. Then it determines the email is new and
If the remote bounces the message back as too large with the full
message attached, this too starts the mail duplication cycle. And
whatever you do, DO NOT delete all these duplicate emails into your
trash folder! That will start the duplicate email cycle on EACH of
the already massively duplicated emails—so yes, it can be triggered
without another MUA putting massively large messages into folders.
Oh and when cleaning up this way, there may still be one or more
copies on the server that will be downloaded and continue the massive
mail duplication you just tried to stop. Without direct access to
the message directory on the server, your best bet for cleanup is to
ctrl-c offlineimap, then kill -9 it because it doesn't terminate
(*sigh*), then nuke the messages by hand in your local folders, then
log into squirrelmail and do the same on the remote folders, and then
hopefully probably it won't do that again.
You can work around the problem by telling offlineimap to skip large
messages, but this really isn't desired. It's what I'm doing at the
moment, but … I am not sure how big the message has to be before
things start getting out of hand.
I'm unable to realistically test this further because having it
happen while I was away for a week brought me too near CenturyLink's
usage cap. And when you cross the threshold, they start throwing
packets on the floor randomly. Incidentally, I will be canceling
their service next week, and I intend to tell them exactly why. If
they give me any crap, the cancel-my-service call will be published.
Other clients (mutt, Mail.app, Thunderbird, etc.) do not exhibit this
More information about the OfflineIMAP-project