storing and synchronizing labels

[Abdó Roig-Maranges]

Hi,


> I also thought about inserting it into message headers, but it seemed too
> invasive to me. I felt that message flags/labels should not influence its
> content.

Yes, that's true, messages are supposed to be immutable. But after giving it a
lot of thought, I realized it was the cleanest solution I could come up with,
and decided it was worth making messages a little mutable.

If you store labels somewhere else, you have to be careful to keep that
somewhere else in sync with the messages, especially if you intend to sync them
across computers, and let mail clients read and change them.

I did try this, and it came up messier than I anticipated at first. Also, if
the email client needs to change labels, it has to know how they are stored
elsewhere, tying it tighter to the particular implementation in offlineimap
label storage.

The X-Keywords thing is not particularly standard, but at least mu/mu4e (the
email client I use) and I think mutt are aware of it to some extent.


> I suppose there were reasons why other mail stores like Maildir keep
> flags separately.  One reason could be that two programs modifying message
> labels concurrently can mess it up.

This is a valid concern. Offlineimap write operations are done "the maildir
way": save the modified file to tmp, and then rename to the original path, which
is an atomic operation (at least on linux).

However, if an external program does the same, at the same time, it can mess up
the copy in tmp. But I think it can be effectively worked around making sure the
copy in tmp does not collide with others, using a unique name, or something like
that.


> Only now I discovered that OfflineIMAP already uses SQLite for storing 
> cache information. So storing messages there would be probably even 
> simpler than I thought.

Offlineimap can use a sqlite database for the cache. As part of my patch, and
for the sake of label syncing, labels are stored in the message cache. This way,
offlineimap can distinguish whether a label has been deleted locally, or added
remotely. To be able to do this we need to keep state information on three
places: locally (the messages), cache, and remotely (gmail).

The cache should not be used to store labels without the "local" side. If you
did only store labels in the cache, and let external applications change those
labels, you would loose track of the state at the moment of the last sync (which
is what the cache tells you). Then you could not distinguish between these two
cases:

  1. A label 'foo' has been added on the gmail side and should propagate to local.
  2. A previously existing label 'foo' has been deleted locally, and should be
     deleted on gmail.

Abdó.