PARTIALLY REMOVING MAXAGE (was: [PATCH v4] make maxage use UIDs to avoid timezone issues)

[Nicolas Sebrecht]

On Fri, Mar 27, 2015 at 11:22:43AM -0400, Janna Martl wrote:
> On Fri, Mar 27, 2015 at 02:54:41PM +0100, Nicolas Sebrecht wrote:

> I'm confused; the IMAP spec says:
> 
> >1) Unique identifiers MUST be strictly ascending in the
> >   mailbox at all times.  If the physical message store is
> >   re-ordered by a non-IMAP agent, this requires that the
> >   unique identifiers in the mailbox be regenerated, since
> >   the former unique identifiers are no longer strictly
> >   ascending as a result of the re-ordering.
> 
> and you told me that this is not the kind of thing that Gmail messes up.

Yes, this remains true. But it is more subtle.

We come to glitches when we upload mails in very short delays.  The
"order" UIDs are assigned won't match the strict order of the uploads.

The thing is that for maxage, what is relevant is really this upload
order. Not the UIDs order only. This feature will work ONLY IF we are
guaranteed that BOTH sides keep the same order of the _messages_.

So to make it simple, the RFCs are respected in the sense that each
side won't mess its own UIDs ascending order. What we are not guaranteed
is that the order of the _messages_ are kept in order _accross_
mailboxes.

I can't find a correct workaround about that:

   (local)               (remote)
  ---------------------------------
   M (UID=100)           N (UID=50)
   N (UID=101)           M (UID=51)

Each side has strict UID ascending orders. Both messages orders don't
match. This is the case you've hit in your tests.

For maxage, we would look for min(UID) of local, find 100. Then work
with the UID=51 on remote. Owned!

> I have a feeling that people are more interested in using the IMAP-IMAP
> thing to copy the contents of one account to a new account. I wonder if
> we could only support using maxage in this case, i.e. when one of the
> accounts is empty. That would take away most of the edge cases, and if
> the messages show up in the other account slightly out of order, it's no
> big deal; it could have happened naturally, and that's exactly what the
> min(uid) thing is good at dealing with in the IMAP-Maildir case.

Makes sense. But in this case, we are sightly diverging from the
definition of maxage. Here, we are taking a UID reference from one side
once which won't anymore be changed.

This is a good idea and this might really fill the gap of a maxage
removal. I'm all for this if we don't hijack the maxage definition.
IOW, this would require yet another configuration option.

Under the hood, relying on the maxage implementation for computation
would be right. Notice we would need to store the initial min(UID) in
the cache.

-- 
Nicolas Sebrecht