[PATCH] More detailed error output on corrupt UID mapping files

[chris coleman]

On Mon, 2 May 2011 07:52:22 -0700 (PDT), chris coleman <christocoleman at yahoo.com> wrote:
> Thanks Sebastian and Nicolas.
> 
> The corrupt data has been found.
> 
> You're right it was indeed in the huge file ~/.offlineimap/Repository-IMAPLocal<username>/UIDMapping/INBOX.Sent  
> 
> Here are a few lines before and after :
> 16731:4777
> 16732:4778
> 16733:4779
> 16734:  P    35:4781
> 16736:4782
> 16737:4783
> 16738:4784
> They're consecutive UIDs so I re-entered them and filled in the missing numbers in the sequence.
> 
> With the INBOX.Sent file repaired in this user, I re-ran it, and we're down to this same basic error on both mailboxes:
> 
> 
> "WARNING: ERROR in syncfolder for username1: null byte in argument for long()"
> "WARNING: ERROR in syncfolder for username2: null byte in argument for long()"
> 
> 
> Do you have a patch for this "ERROR in syncfolder", to pinpoint the line number and filename ?

The error most likely occurs in offlineimap/offlineimap/folder/IMAP.py
around line 162 in the cachemessagelist() function

What happens is that we select a folder in line 111 in:

imaptype, imapdata = imapobj.select(self.getfullname(), readonly = 1, force = 1)

and later in line 162 we go through the returned data, trying to get the
highest UID number that the select call returned:

                for msgid in imapdata:
                    maxmsgid = max(long(msgid), maxmsgid)

If you could add some debug code in the form

                for msgid in imapdata:
                    try:
                        maxmsgid = max(long(msgid), maxmsgid)
                    except:
                        raise Exception(str(imapdata))

and let us see what exactly had been returned in imapdata we could try
to work this out.

Let me know if you need more help in making the code changes. I hope we
get this one nailed down.

Sebastian

============

I'm on 6.3.2-0 because it's the highest that will run on debian 5.0 (lenny).

The catch is, 6.3.2-0 code for offlineimap/offlineimap/folder/IMAP.py is using imapobj not imapdata - so it seems like your patch would need modification to fit.

Here is the code excerpted from lines 175-188:

                try:
                    # 1. Some mail servers do not return an EXISTS response
                    # if the folder is empty.  2. ZIMBRA servers can return
                    # multiple EXISTS replies in the form 500, 1000, 1500,
                    # 1623 so check for potentially multiple replies.
                    maxmsgid = 0
                    for msgid in imapobj.untagged_responses['EXISTS']:
                        maxmsgid = max(long(msgid), maxmsgid)
                    messagesToFetch = '1:%d' % maxmsgid;
                except KeyError:
                    return
                if maxmsgid < 1:
                    #no messages; return
                    return

Your suggestion on how to patch this ?

Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/offlineimap-project/attachments/20110502/2699f446/attachment-0001.html>