Strange behavior when syncing two IMAP servers

[Sebastian Spaeth]

On Thu, 24 Nov 2011 03:05:56 +0100, Matthieu Rakotojaona wrote:
> I am working on an IMAP server of my own. I'm trying to sync it with a
> GMail account, with a nametrans.
> 
> System : Linux 3.1.1, OfflineIMAP v.6.3.4
> 
> When I add a message to a mailbox on my side and want to sync
> (upstream), here is the behavior I see on my side, on the IMAP server:
> 
> 
> So, I have a few questions :
> 
> * My custom server cannot append a message if it already exists. Why
> does it append exactly the same, to remove the first one ?
> All I can see from this is that the email should get a new UID, but I
> can't see the reason. And it doesn't serves my purpose =]
> 
> * Where can I see the relevant parts (2, 3 ( I suppose OfflineIMAP
> verifies some things at that moment), and 4) in the code ? I couldn't
> find them.

Hi Matthieu,

interesting to see that sequence :-). First of all: offlineimap has
originally been designed for IMAP<->Maildir synchronization, and
IMAP<->IMAP has been bolted on later. And that shows, IMAP<->IMAP does
some (very) suboptimal stuff.

> 1) SELECT, EXAMINE, FETCH FLAGS, SELECT, EXAMINE the mailbox

This would be getting a list of Mails & flags on that folder, not sure
why the send SELECT is there. this is the cachemessagelist() part in
folder.UIDMaps.py

Everything concerned with uploading the mail starts from folder.Base.py
in the copymessageto() function.

> 2) FETCH BODY.PEEK[] the new email

Well obviously reading in the mail via getmessage()

> 3) SELECT the mailbox
> 4) APPEND the exact same email
> 5) CHECK, SELECT the mailbox
> 6) DELETE (STORE +FLAGS \Deleted) the email (the one that was read,

THe above is clearly the suboptimal part, and this is what happens in
copymessageto():

# Got new UID, change the local uid.
#TODO: Maildir could do this with a rename rather than
#load/save/del operation, IMPLEMENT a changeuid()
#function or so.
self.savemessage(newuid, message, flags, rtime)
self.deletemessage(uid)

We save the message to record a new UID that we have received (this will
happen when the other side is an IMAP server) and then delete the message with
the old ID. In the case of a MappedIMAP server, all this serves to is to
record the UID mapping between the upstream and the local IMAP server.

Clearly this is less than optimal and as you can see, I have included
the TODO comment that we should implement. I have a branch that
implements that, but more critical things had always popped up.

The only excuse I have is to say, it has always worked this way even
before I came onboard :-). And this did not hurt as much in the Maildir
case, where a load/save (while inefficient) does not hurt as much as it
does in the IMAP case.

> not the one that was appended)
> 7) SELECT, EXPUNGE the mailbox

That is when closing the connection(?)

Sebastian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/offlineimap-project/attachments/20120106/d6094a52/attachment.sig>