<DKIM> Re:<DKIM> AssertionError in cachemessagelist

[Isaac Bennetch]

Hello,

On 12/22/14, 8:59 PM, Nicolas Sebrecht wrote:
> On Mon, Dec 22, 2014 at 02:04:09PM -0500, Isaac Bennetch wrote:
> 
>>> Please, show us the head of your LocalStatus files. They stands in the
>>> cache. Look for the Account-<name> with the name of the offending
>>> account:
>>>
>>>   % cd <cache path>/Account-<name>/Localstatus
>>>   % file *
>>
>> [Gmail].All Mail: ASCII text
> 
> Hmm. I think I'm going to have a good understanding of what's going
> wrong and I wonder if the file utility is fair play by pretending it's
> ASCII text.
> 
> I'm pretty sure this cache file has at least one non-ascii character
> somewhere so that python str assert() is puzzled with a unicode type.
> 
> I bet it comes from a Gmail label. One way to fix it might be to come
> back to ASCII labels. It SHOULD work. Please, don't. I have more
> extra-work for you.

I'm happy to do extra work to test and fix this :)

> If I'm write, here is a patched version to fix encodings handling.
> Download the zip at
> 
>   https://github.com/nicolas33/offlineimap/archive/fix-unicide-localstatus.zip
> 
> unzip and
> 
>   % cd offlineimap-fix-unicide-localstatus
>   % ./offlineimap # with your usual options
> 
> The patch I wrote introduce support for non full ASCII cache file.

I have done so and it appears to work correctly -- both accounts are
downloading many old messages and I do not see the error message.

This is particularly odd to me because, as far as I can tell, I do not
have any unicode characters in my Gmail labels. Of course, I could be
wrong, but nothing is obvious when looking through the list.

> I did check that all type assumptions regarding enconding are valid
> LOCALLY in the source file.  It should be reasonable to assume it's
> enough. I guess that there is only one driver to this cache file but I
> didn't check the whole code in all the source files. So, I can't be 100%
> sure the patch is enough.
> 
> As a consequence, please MAKE a full backup of your cache and your mails
> before EACH sync while testing.
> 
>>>   % head -v -n1 *
>>
>> OFFLINEIMAP LocalStatus CACHE DATA - DO NOT MODIFY - FORMAT 2
> 
> So, the header does not look like plain broken. Good.
> 
>> I get the same output on both of my misbehaving accounts.
>>
>> I've noticed a discrepancy -- running head on the account that works
>> properly gives
>>
>> OFFLINEIMAP LocalStatus CACHE DATA - DO NOT MODIFY - FORMAT 1
>>
>> Any thought about why the working accounr is FORMAT 1 and both
>> non-working accounts are FORMAT 2?
> 
> Interresting. The format number is _to be_ updated to 2 since
> OfflineIMAP v6.5.6-rc1. I say "to be", because the cache file is updated
> only if it needs to be updated, regardless the current version.
> 
> The code that updates the file format from 1 to 2 and the offending
> assert() were both introduced by the same commit. So, I know you have at
> least v6.5.6-rc1 (or above 09556d6 to be exact). Otherwise, you would
> never hit that "offending" assert().
> 
> I think that the working account cache file was not updated because
> OfflineIMAP never had to update/rewrite it since you're above
> v6.5.6-rc1.
> 
> IOW, it's fine to have cache files not at the exact same version.

Thank you for the detailed explanation on cache files as well as the
unbelievably fast patch. Is there some further testing I can do?

Regards,
~isaac