problem upgrading from offlineimap 6.3.4 to 6.6.1

[Ilias Tsitsimpis]

On Fri, Mar 04, 2016 at 12:11PM, Nicolas Sebrecht wrote:
> On Fri, Mar 04, 2016 at 09:50:16AM +0100, francois at avalenn.eu wrote:
> > On Sun, Feb 28, 2016 at 07:29:02PM +0100, Nicolas Sebrecht wrote:
> 
> > > I guess this might be confusing or make bad things (including data loss)
> > > if applied on mails that has moved from a folder to another one without
> > > a full filename change because this might lead to UID conflicts.
> > 
> > Indeed at first glance patch seems sound but this potential UID
> > conflicts annoys me. Is it possible to retrieve the older FMD5 (before
> > the change of using nametrans) and replace only those matching the old
> > one ?
> 
> Theorically, I believe that's possible to re-introduce the old algo to
> re-hash the old FMD5.

IIRC the old algo was using the local folder name in order to compute
the FMD5. So, it is relatively easy to check if the current FMD5 matches
that of the local folder name and only then change it to the new one.
This will indeed provide extra insurance against potential UID
conflicts, but at the same time limit the patch to only those cases
where one needs to upgrade from a version older than v6.3.5. I still
cannot find another usage for the above patch (i.e., other cases where
FMD5 has been modified and needs fixing), so, if you like, we may go
that way (and change the name of the flag to something like
--migrate-fmd5-using-nametrans).

If we choose to leave it as is, I will make sure that at least all
messages in the folder have the same FMD5 before renaming them
(otherwise we end up with UID conflicts for sure).

> However, I'm not much worried. The conflicts would only appear in the
> very edge-case when the user runs the fix while unexpected filemane
> changes happened since the previous sync. I'd say this edge-case would
> not worth the trouble in the code. If you worry about that, it's enough
> to add a warning before fixing the hashes. Something like:
> 
>   "Ensure to dispose a proper backup of both the cache and the Maildir
>   before running this fix. Be aware that if you made unexpected changes
>   to filenames in the maildir (behind OfflineIMAP's back) since your
>   previous sync, bad things might occur.
> 
>   Continue? (y/n)"

Indeed, the edge-case will occur if the user happens to move a message
since the previous sync. But as rare as this may seems, the results are
very bad (UID conflicts) and users are very sensitive with their emails.
I understand that there is no algorithm that will work for everyone,
hence, we will make sure that they have read the blog-post and made a
backup.

> Either way would still be much better than the current alternatives
> where users have to find the proper fix by themselves and possibly do
> worse things.
> 
> Also, I may update the blog post written by François at
> 
> http://www.offlineimap.org/configuration/2016/02/12/debian-upgrade-from-jessie-to-stretch.html
> 
> if none of you intend to do it.

I will update the patch and send a pull request (hopefully by the weekend).

Cheers,
Ilias