HEUREKA, Missing mails log snippet (found it)

[Piers Lauder]

On Wed, 08 Jun 2011 16:08:44 +0200, Sebastian Spaeth wrote:
  > 
  > A crime confessed is half redressed...
  > 
  > Dear offlineimap list, Dear Peter, Dear Piers.
  > 
  > GOTCHA! The case of trying to delete mails is solved. Thanks to logs
  > from Peter Hutterer, I was able to find out what going wrong.
  > 
  > We fetch a list of UIDs from the server, but the list must currently not
  > be interrupted with unrelated responses.
  > 
  > THIS is send right after one of "missing UIDs"
  > 
  > DEBUG[imap]:   26:17.31 who-t.net reader < * OK Searched 43% of the mailbox, ETA 0:12\r\n
  > 
  > 
  > so this series (handler log):
  > untagged_responses[FETCH] 10821 += ["10822 (FLAGS (\Answered \Seen) UID 27260)"]
  > untagged_responses[OK] 0 += ["Searched 43% of the mailbox, ETA 0:12"]
__^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(Just as I suspected :-)

  > untagged_responses[FETCH] 0 += ["10823 (FLAGS (\Answered \Seen) UID 27261)"]
  > 
  > is broken into several "Responses" (note the FETCH restarts counting at
  > 0). It breaks the series of untagged_responses just as Piers had
  > suspected. Piers, what's the recommended fix? 
  > 
  > Look through ALL untagged_responses as you had suggested? Is this
  > something we should fix at offlineimap level or do you want to fix this
  > on imaplib2 level and we bundle the new imaplib2 asap?
  > 
  > This is a pretty bad case of data loss for us, so it should be high
  > priority.
  > 
  > Sebastian

The fix is that piece of code I sent you yesterday (and pushed to
sourceforge), rewriting _untagged_response(<name>) to grab _all_ <name>
responses, repeated below:

    def _untagged_response(self, typ, dat, name):

        if typ == 'NO':
            return typ, dat
        data = self._get_untagged_response(name)
        if not data:
            return typ, [None]
        while True:
            dat = self._get_untagged_response(name)
            if not dat:
                break
            data += dat
        return typ, data