<DKIM> Re: moving mails between directories does not work

[Nicolas Sebrecht]

On Mon, Nov 06, 2017 at 11:24:44PM +0100, Jan Knížek wrote:
> >
> >Ok. Would be good to have the logs of all 3 runs with imap debug mode
> >enabled ('-d imap').
> >
> 
> Ok. Everything is run with `offlineimap -1 -d imap`.
> 
> debug1 - when A in mbox first appeared
> debug2 - in this sync it gets replicated
> debug3 - before this sync local-mbox had been emptied, so after this there
>             is exactly one A

Ok, thank you. Here's what I get from the logs of debug1:

1. Offlineimap starts the process to copy the new email UID -1 to the remote.

2. Because it's missing UIDPLUS, it's adding the X-OfflineIMAP header.

3. The email gets copied but the newly assigned UID is missing (expected
because there's no UIDPLUS).

4. Offlineimap tries to get the UID of this message via the
   X-OfflineIMAP header. There are two methods.

   a. Tries the first method but fails because the connection was closed
   by the remote.

   b. Tries the second method but fails for the same reason.

IOW, offlineimap APPENDs the email but failed to get the new UID to
update the database (and maildir). Hence, the email is replicated on
next run (it's downloaded as a new email from the remote since it could
not link it to the local mail with the UID).


So, the remote is wrongly closing the socket just after the APPEND.
Offlineimap fails at handling the socket closed (it could try to repoen
it). Hence, it fails at assigning the UID and reports the "connection
closed" message. However, this message did not happen while uploading
the mail but while trying to figure the UID by our means.


I guess we didn't have other issues like this because such context
is likely rare:

  - the IMAP server doesn't support UIDPLUS. (I think it's rare those
    days while ok.)

  - the IMAP server closes the socket after the APPEND of the new email.
    (It should not, why the server does this? It's not IMAP compliant.)


I'll improve the logs. The current reports are not good enough to figure
what's happening. For now, we wrongly let the user think that the APPEND
failed while it likely worked.

I'll see if I can improve the behaviour of offlineimap to recover from
this. I'm not sure I'll have the time to try making offlineimap support
a non-IMAP-compliant provider. This would require you to test patches,
too.

-- 
Nicolas Sebrecht