[PATCH] More detailed error output on corrupt UID mapping files

[Sebastian Spaeth]

On Mon, 2 May 2011 07:52:22 -0700 (PDT), chris coleman <christocoleman at yahoo.com> wrote:
> Thanks Sebastian and Nicolas.
> 
> The corrupt data has been found.
> 
> You're right it was indeed in the huge file ~/.offlineimap/Repository-IMAPLocal<username>/UIDMapping/INBOX.Sent  
> 
> Here are a few lines before and after :
> 16731:4777
> 16732:4778
> 16733:4779
> 16734:  P    35:4781
> 16736:4782
> 16737:4783
> 16738:4784
> They're consecutive UIDs so I re-entered them and filled in the missing numbers in the sequence.
> 
> With the INBOX.Sent file repaired in this user, I re-ran it, and we're down to this same basic error on both mailboxes:
> 
> 
> "WARNING: ERROR in syncfolder for username1: null byte in argument for long()"
> "WARNING: ERROR in syncfolder for username2: null byte in argument for long()"
> 
> 
> Do you have a patch for this "ERROR in syncfolder", to pinpoint the line number and filename ?

The error most likely occurs in offlineimap/offlineimap/folder/IMAP.py
around line 162 in the cachemessagelist() function

What happens is that we select a folder in line 111 in:

imaptype, imapdata = imapobj.select(self.getfullname(), readonly = 1, force = 1)

and later in line 162 we go through the returned data, trying to get the
highest UID number that the select call returned:

                for msgid in imapdata:
                    maxmsgid = max(long(msgid), maxmsgid)

If you could add some debug code in the form

                for msgid in imapdata:
                    try:
                        maxmsgid = max(long(msgid), maxmsgid)
                    except:
                        raise Exception(str(imapdata))

and let us see what exactly had been returned in imapdata we could try
to work this out.

Let me know if you need more help in making the code changes. I hope we
get this one nailed down.

Sebastian

> 
> 
> Chris
> 
> 
> 
> 
> =============
> 
> This function will need much more "robustifying", but the very least we
> can do is to print the file name and line that are giving trouble.
> 
> Signed-off-by: Sebastian Spaeth <Sebastian at SSpaeth.de>
> ---
> Chris, you might want to try this patch to see the exact file name and line
> that are giving you trouble. Nicolas, I advocate this one for going into master :-)
> 
> Changelog.draft.rst           |    1 +
> offlineimap/folder/UIDMaps.py |    6 +++++-
> 2 files changed, 6 insertions(+), 1 deletions(-)
> 
> diff --git a/Changelog.draft.rst b/Changelog.draft.rst
> index d0125b7..11d16ee 100644
> --- a/Changelog.draft.rst
> +++ b/Changelog.draft.rst
> @@ -18,6 +18,7 @@ Changes
> 
> - no whitespace is stripped from comma-separated arguments passed via
>    the -f option.
> +- give more detailed error when encountering a corrupt UID mapping file
> 
> Bug Fixes
> ---------
> diff --git a/offlineimap/folder/UIDMaps.py b/offlineimap/folder/UIDMaps.py
> index 43b28e4..e87e21e 100644
> --- a/offlineimap/folder/UIDMaps.py
> +++ b/offlineimap/folder/UIDMaps.py
> @@ -51,7 +51,11 @@ class MappingFolderMixIn:
>                  line = file.readline()
>                  if not len(line):
>                      break
> -                line = line.strip()
> +                try:
> +                    line = line.strip()
> +                except ValueError:
> +                    raise Exception("Corrupt line '%s' in UID mapping file '%s'" \
> +                                        %(line, mapfilename))
>                  (str1, str2) = line.split(':')
>                  loc = long(str1)
>                  rem = long(str2)
> -- 
> 1.7.4.1Non-text part: text/html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/offlineimap-project/attachments/20110502/f8ec9d9e/attachment.pgp>