[PATCH] Syncing gmail labels for gmail accounts

Thu Nov 22 15:46:15 GMT 2012

Hi,

in Python documentation for re.sub() you can find: "Changed in version 3.1:
Added the optional flags argument". So the code in this patch will work
only with Python 3.1+. I did not look in the code, just the stack trace you
posted, but I guess we can do without using the optional "flags" in re.sub,
can't we? Then the patch would be compatible with older Python versions.

Best regards,
Tomek

2012/11/22 Dima Pasechnik <dimpase+olimap at gmail.com>

> Hi,
> I am trying this patch on my gmail account
> (specifically, the code I use is here:
> https://github.com/dimpase/offlineimap)
> and get the following errors (which look like a bug being triggered):
>
> $ offlineimap
> OfflineIMAP 6.5.5-rc2
>   Licensed under the GNU GPL v2+ (v2 or any later version)
> Account sync main:
>  *** Processing account main
>  Establishing connection to imap.gmail.com:993
> Folder [Google Mail]/All Mail [acc: main]:
>  Syncing [Google Mail]/All Mail: Gmail -> GmailMaildir
>  Collecting data from messages on [Google Mail]/All Mail
>  Copy message 211570 (1 of 3) gmail-remote:[Google Mail]/All Mail ->
> main-local
>  Copy message 211572 (2 of 3) gmail-remote:[Google Mail]/All Mail ->
> main-local
>  Copy message 211571 (3 of 3) gmail-remote:[Google Mail]/All Mail ->
> main-local
> Copy message from gmail-remote:[Google Mail]/All Mail:
>  Establishing connection to imap.gmail.com:993
>  Establishing connection to imap.gmail.com:993
> Folder [Google Mail]/All Mail [acc: main]:
>  Setting labels to message 211466 on [Google Mail].All Mail (1 of 4):
>  ERROR: Syncing folder [Google Mail]/All Mail [acc: main]
>   sub() got an unexpected keyword argument 'flags'
>  ERROR: ERROR in syncfolder for main folder [Google Mail]/All Mail:
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.6/dist-packages/offlineimap/accounts.py",
> line 446, in syncfolder
>     remotefolder.syncmessagesto(localfolder, statusfolder)
>   File "/usr/local/lib/python2.6/dist-packages/offlineimap/folder/Base.py",
> line 642, in syncmessagesto
>     action(dstfolder, statusfolder)
>   File
> "/usr/local/lib/python2.6/dist-packages/offlineimap/folder/Gmail.py",
> line 328, in syncmessagesto_labels
>     dstfolder.savemessagelabels(uid, selflabels, ignorelabels =
> self.ignorelabels)
>   File
> "/usr/local/lib/python2.6/dist-packages/offlineimap/folder/GmailMaildir.py",
> line 150, in savemessagelabels
>     content = self.message_addheader(content, self.labelsheader,
> labels_str)
>   File "/usr/local/lib/python2.6/dist-packages/offlineimap/folder/Base.py",
> line 385, in message_addheader
>     leader, flags = re.MULTILINE)
> TypeError: sub() got an unexpected keyword argument 'flags'
>
>   sub() got an unexpected keyword argument 'flags'
> Account sync main:
>  *** Finished account 'main' in 0:45
> ERROR: Exceptions occurred during the run!
> ERROR: Syncing folder [Google Mail]/All Mail [acc: main]
>   sub() got an unexpected keyword argument 'flags'
> ERROR: ERROR in syncfolder for main folder [Google Mail]/All Mail:
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.6/dist-packages/offlineimap/accounts.py",
> line 446, in syncfolder
>     remotefolder.syncmessagesto(localfolder, statusfolder)
>   File "/usr/local/lib/python2.6/dist-packages/offlineimap/folder/Base.py",
> line 642, in syncmessagesto
>     action(dstfolder, statusfolder)
>   File
> "/usr/local/lib/python2.6/dist-packages/offlineimap/folder/Gmail.py",
> line 328, in syncmessagesto_labels
>     dstfolder.savemessagelabels(uid, selflabels, ignorelabels =
> self.ignorelabels)
>   File
> "/usr/local/lib/python2.6/dist-packages/offlineimap/folder/GmailMaildir.py",
> line 150, in savemessagelabels
>     content = self.message_addheader(content, self.labelsheader,
> labels_str)
>   File "/usr/local/lib/python2.6/dist-packages/offlineimap/folder/Base.py",
> line 385, in message_addheader
>     leader, flags = re.MULTILINE)
> TypeError: sub() got an unexpected keyword argument 'flags'
>
>   sub() got an unexpected keyword argument 'flags'
> $
>
> Would be great is this is easily fixable, as I really like the extras
> this patch
> is offering.
>
> Best,
> Dmitrii
>
> On 26 October 2012 02:33, Abdó Roig-Maranges <abdo.roig at gmail.com> wrote:
> >
> > Hi,
> >
> > I've been trying to find a good way to sync gmail labels (and use them
> > in the email client) for some time now. My system up until now involved
> > syncing labels into an external file, and using a lot of glue scripts to
> > keep them in sync, update notmuch tags, etc. Very messy.
> >
> > I've now decided to try a different approach, which seems pretty close
> > to the right one. I've added support in offlineimap to sync gmail
> > labels, and store them inside the messages, under the header
> > X-Keywords. This way, the labels "move with the messages" (even if I
> > copy the maildir to an other machine, etc etc).
> >
> > There are some MUA that even recognize to some degree the X-Keywords
> > header or a close relative, X-Label [2]. It seems mutt supports X-Label
> > [3] and mu supports both [4].
> >
> > I've been happily using this approach, with mu/mu4e as my MUA / indexer,
> > for a week now. So here I attach a couple of patches (against current
> > master) which do the following (see the individual commits for more
> > details).
> >
> > bugfixes.patch :
> >
> >   1. fixes a couple of easy bugs I found. It must be applied first, and
> >      is independent of the rest.
> >
> >
> > gmail-labels.patch:
> >
> >   2. When a message goes out of gmail, adds a header X-Keywords with a
> >      comma separated list of labels.
> >
> >   3. Updates the LocalStatus Sqlite table to include columns for labels
> >      and local mtimes. For non-gmail repositories these columns are
> >      ignored.
> >
> >   4. When labels change on the gmail side, syncs them the same way as
> >      flags get synced (comparing with LocalStatus etc)
> >
> >   5. Adds a GmailMaildir folder type, which keeps track of individual
> >      message modification times (the POSIX mtime), and uses it to spot
> >      messages which have been modified locally. Then, only for those
> >      modified messages (typically very few), reads the labels and syncs
> >      them back to gmail, the same way as flags.
> >
> >   6. Adds an option to filter out certain headers when uploading
> >      messages to gmail. One may want to remove X-Keywords before sending
> >      a message back to gmail.
> >
> >   7. Adds an option to ignore certain labels, like \Draft, for which
> >      flags serve the same purpose. Gmail internally keeps the D flag in
> >      sync with the \Draft label, or the F flag with the \Starred label.
> >
> >
> > Some comments:
> >
> >   1. These changes (which are quite a few) should interfere minimally
> >      with non-gmail users. The only exception is the update on
> >      LocalStatus sqlite table, which I hope will cause little trouble.
> >
> >   2. There are some issues with the SQLite backend. Right now, it
> >      commits to the database too frequently IMHO (after every message
> >      copy). This produces a lot of disk activity. I may look into
> >      it... my approach would be storing the status in memory during the
> >      message copying and commit to database once, at the end. Any
> >      thoughts? I don't think there is danger of losing data, on
> >      crashes. The LocalStatus will be updated correctly on the next run.
> >
> >   3. The slower part is folder.Gmail.cachemessagelist, that downloads
> >      the uids and labels for all messages. It takes about 17 seconds
> >      with about 25k moderately labelled messages. I have done some
> >      experiments with multiple threads without improvement. My guess is
> >      that on the gmail side there is some sort of bandwith
> >      throttling. Not even compressing the connection improves matters.
> >
> > Well, that's it. I send this patch hoping some developer here may take
> > the time to look at it. Being able to use and sync labels with the web
> > interface or mobile would be a very nice addition for which, as far as I
> > know, there is no alternative solution out there.
> >
> >
> > [1] http://comments.gmane.org/gmane.mail.imap.offlineimap.general/5916
> > [2] http://does-not-exist.org/mail-archives/mutt-dev/msg08249.html
> > [3] http://blitiri.com.ar/p/other/mutt-labels/
> > [4] https://github.com/djcb/mu/issues/40
> >
> > Abdó.
> >
> >
> > _______________________________________________
> > OfflineIMAP-project mailing list
> > OfflineIMAP-project at lists.alioth.debian.org
> >
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/offlineimap-project
> >
> > OfflineIMAP homepage: http://software.complete.org/offlineimap
>
> _______________________________________________
> OfflineIMAP-project mailing list
> OfflineIMAP-project at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/offlineimap-project
>
> OfflineIMAP homepage: http://software.complete.org/offlineimap
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/offlineimap-project/attachments/20121122/b249dcfd/attachment-0003.html>