Seeing lots of Key Error messages lately

[Sebastian Spaeth]

On Thu, 09 Feb 2012 16:16:37 -1000, John Wiegley <jwiegley at gmail.com> wrote:
> Sebastian,
> 
> I upgraded my spaetz/next a few days ago, to 7f22d89, and I'm now seeing lots
> of KeyErrors:
> 
> --8<---------------cut here---------------start------------->8---
> Deleting 10 messages (4419:4428) in MappedIMAP[Lists.WG21.Reflector]
> ERROR: Syncing folder Lists/WG21/Reflector [acc: Gmail]
>   4420
> ERROR: ERROR in syncfolder for Gmail folder Lists/WG21/Reflector: Traceback (most recent call last):
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/offlineimap/accounts.py", line 431, in syncfolder
>     remotefolder.syncmessagesto(localfolder, statusfolder)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/offlineimap/folder/Base.py", line 474, in syncmessagesto
>     action(dstfolder, statusfolder)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/offlineimap/folder/Base.py", line 386, in syncmessagesto_delete
>     folder.deletemessages(deletelist)
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/offlineimap/folder/UIDMaps.py", line 277, in deletemessages
>     self._mb.deletemessages(self._uidlist(self.r2l, uidlist))
>   File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/offlineimap/folder/UIDMaps.py", line 85, in _uidlist
>     return [mapping[x] for x in items]
> KeyError: 4420

This is really a corruption of the UIDmaps list which will not repair
itself (it can not lead to data loss, but only to message
duplication). I really need to implement a "repair" mode that 1) deletes
the corrupt mapping, and 2) repairs the mapping file by comparing the
messages in both IMAP folders.
1) is quite easy, 2) is harder but doable.

Let me try to implement 1), so that you get sane UID maps back (at the
cost of a few duplicated messages).

> I'm also seeing messages come back as
> "unread" in my mailing lists, even though I remember marking them as read.

Yep, that could be a different case, and being something gmail
specific...
The issue is that folder moves are implemented as a 1) delete in folder
A and 2) append to folder B. Now if you delete a message from a folder
(so that it is just left in "All Mails" and then add it to another
folder, it might get set back to "unread" status by gmail)

Sebastian