Sync dovecot to cyrus imap, creates many X-OfflineIMAP headers

[chris coleman]

Sebastian Spaeth wrote:
On Wed, 17 Aug 2011 11:00:48 -0700 (PDT), chris coleman wrote:
> On syncs today, OfflineIMAP does things like this: 
> When I move a message, on the local server, from the Inbox to the Trash, offlineimap deletes the Inbox message copy on the remote server, and uploads the message from local server Trash to remote server Trash !  

Yep, that is the expected operation.

> For a 1MB message, that's 10 seconds to upload to the remote server's Trash!  
> 
> OfflineIMAP would run 20X faster/more efficiently if it would *move* the message from the remote server Inbox, to the remote server Trash - a move operation should execute in about 0.5 second.

It would but a) it operates on a folder-by-folder basis (no excuse
though) and b) it is not trivial to detect such moves between
folders. It is not uncommon to have multiple emails with the same
Message-ID in different folders, and they do not necessarily have the
same content. Think e.g. a mail that you sent and that you get back via
a mailing list subscription. It will have the same Message-ID, it will
be in your "Sent" folder and it will be in your "INBOX" folder. Some
broken MUAs are also said to produce identical Message-IDs from time to
time.

Detecting that you actually moved a mail from one folder to another is
therefore no trivial business. The safe bet is to treat these as a
removal and addition in the folders.

Sebastian
------------------


Hi Sebastian,

With UID/UIDPLUS isn't it now possible to save the list of UID's for each folder, in the sqlite database, at the end of the current run of OfflineIMAP ?

At the start of the next run, compare the UIDs in each folder to the previously seen list of UIDs for each folder ?

This method lets us track movements of messages between the folders, ie when a message is not "new in a folder but we don't know where it came from" just "this message was simply moved from Inbox to Trash"

So, with the saved information about where the message came from, we are able to greatly improve the efficiency of the algorithm: just do a 0.5 second MOVE on the remote IMAP server instead of consuming 10 seconds of saturated uplink bandwidth.  Less network traffic, less potential for data corruption, faster runtime, less memory and CPU usage, less vulnerable to crashes...

Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/offlineimap-project/attachments/20110818/cf1ae5bc/attachment-0002.html>