[PATCH] Syncing gmail labels for gmail accounts

[Abdó Roig-Maranges]

Hi,

I've been trying to find a good way to sync gmail labels (and use them
in the email client) for some time now. My system up until now involved
syncing labels into an external file, and using a lot of glue scripts to
keep them in sync, update notmuch tags, etc. Very messy.

I've now decided to try a different approach, which seems pretty close
to the right one. I've added support in offlineimap to sync gmail
labels, and store them inside the messages, under the header
X-Keywords. This way, the labels "move with the messages" (even if I
copy the maildir to an other machine, etc etc).

There are some MUA that even recognize to some degree the X-Keywords
header or a close relative, X-Label [2]. It seems mutt supports X-Label
[3] and mu supports both [4].

I've been happily using this approach, with mu/mu4e as my MUA / indexer,
for a week now. So here I attach a couple of patches (against current
master) which do the following (see the individual commits for more
details).

bugfixes.patch :

  1. fixes a couple of easy bugs I found. It must be applied first, and
     is independent of the rest.


gmail-labels.patch:

  2. When a message goes out of gmail, adds a header X-Keywords with a
     comma separated list of labels.

  3. Updates the LocalStatus Sqlite table to include columns for labels
     and local mtimes. For non-gmail repositories these columns are
     ignored.

  4. When labels change on the gmail side, syncs them the same way as
     flags get synced (comparing with LocalStatus etc)

  5. Adds a GmailMaildir folder type, which keeps track of individual
     message modification times (the POSIX mtime), and uses it to spot
     messages which have been modified locally. Then, only for those
     modified messages (typically very few), reads the labels and syncs
     them back to gmail, the same way as flags.

  6. Adds an option to filter out certain headers when uploading
     messages to gmail. One may want to remove X-Keywords before sending
     a message back to gmail.

  7. Adds an option to ignore certain labels, like \Draft, for which
     flags serve the same purpose. Gmail internally keeps the D flag in
     sync with the \Draft label, or the F flag with the \Starred label.


Some comments:

  1. These changes (which are quite a few) should interfere minimally
     with non-gmail users. The only exception is the update on
     LocalStatus sqlite table, which I hope will cause little trouble.

  2. There are some issues with the SQLite backend. Right now, it
     commits to the database too frequently IMHO (after every message
     copy). This produces a lot of disk activity. I may look into
     it... my approach would be storing the status in memory during the
     message copying and commit to database once, at the end. Any
     thoughts? I don't think there is danger of losing data, on
     crashes. The LocalStatus will be updated correctly on the next run.

  3. The slower part is folder.Gmail.cachemessagelist, that downloads
     the uids and labels for all messages. It takes about 17 seconds
     with about 25k moderately labelled messages. I have done some
     experiments with multiple threads without improvement. My guess is
     that on the gmail side there is some sort of bandwith
     throttling. Not even compressing the connection improves matters.

Well, that's it. I send this patch hoping some developer here may take
the time to look at it. Being able to use and sync labels with the web
interface or mobile would be a very nice addition for which, as far as I
know, there is no alternative solution out there.


[1] http://comments.gmane.org/gmane.mail.imap.offlineimap.general/5916
[2] http://does-not-exist.org/mail-archives/mutt-dev/msg08249.html
[3] http://blitiri.com.ar/p/other/mutt-labels/
[4] https://github.com/djcb/mu/issues/40

Abdó.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: bugfixing.patch
Type: text/x-patch
Size: 7966 bytes
Desc: bugfixing
URL: <http://alioth-lists.debian.net/pipermail/offlineimap-project/attachments/20121025/4cb06436/attachment-0004.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gmail-labels.patch
Type: text/x-patch
Size: 79518 bytes
Desc: gmail-labels
URL: <http://alioth-lists.debian.net/pipermail/offlineimap-project/attachments/20121025/4cb06436/attachment-0005.bin>