Gmail to "gmail apps" sync: ValueError: Backend could not find uid for message

[Sebastian Spaeth]

(Taking the @googlers off the CC as this is likely beyond their interest)

>  JUST BEFORE the various places in the offlineimap code where a UID might be received as part of a response.  Then, depending on the response, the code would branch between different code paths: the UID case and the no-UID case.

We (offlineimap) ALREADY DO dynamically detect UID responses on APPEND RIGHT
NOW. So we currently cope with APPENDUID changing, should that ever happen.

> Callback based model is one way to call it. 
> I was going to say, Event-driven model, or Message-loop.  

This is now firmly imaplib2-dev land and not offlineimap land anymore
:-) and I will happily defer any design decisions to imaplib2 devs :).

> According to protocol, you've got to respond to IMAP server responses, more or less as they come in (sometimes defer, like when in the middle of a big message or whatever). 
> This is why the Gmail issue happened, with us missing the UIDPLUS
>  capabilities response completely - it actually violates the protocol to look ONLY at the responses that arrive when you assume that they should come in, and ignore the untagged responses.

I agree that it would make sense to cache any incoming CAPABILITY
responses and use these. In fact it would relieve imaplib2 from
explicitely querying them in case the server sends them automatically (as
most seem to do).

The other side of the coin is performance, and that trade-off would have
to be carefully measured, in my opinion. If it slows down ALL IMAP
operations by 10%, I would not like to accept this increased
flexibility for a very rare and unlikely case, but I said already that
I am not the imaplib2 technology architecture officer :-)

> What I meant was, we (offlineimap and imaplib2) shouldn't be working off cached copies of these capabilities values - because IMAP protocol implies that they can and DO go stale.  See example where we crashed on Gmail changed capabilities. 

We never crashed, we just missed out the changed capabilities. :) The case
of data loss came because imaplib2 considered responses interrupted by
an unexpected OK as 2 distinct responses and we were only looking at the
results from the last response. That's definitely an imaplib2 bug, but
IMHO it was just that, a bug :).

> The offlineimap code should be working off the TRUE values, which are subject to change, and should be stored/cached ONLY in imaplib2.  

Again, makes sense, if performance trade-off is seen as
acceptable. Piers is pretty responsive, and I bet he is willing to take
patches. I would be happy if I could focus on offlineimap work, there's
plenty off bigger fish to fry. (amazing how many issues you can detect
in 4k LOC of code)

> Character sets that causes crashes/lost data/logic failures...  this has bitten us in the a** before, and will probably do it again many more times in the future, unless we get a firm grip on it once and for all...

Definitely, and it's always going to be an issue
unfortunately. The use of encodings is one area where I would love
someone would audit OfflineImap carefully for. We are happily mixing
(UTF-8) strings, byte streams and whatnot, hoping for the best...


Sebastian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/offlineimap-project/attachments/20110630/5052770a/attachment.pgp>