<DKIM> Re:[offlineimap] Duplicates of messages copied from remote to local (#174)

[Janna Martl]

On Fri, Jul 24, 2015 at 10:19:13PM +0100, Peter Lewis wrote:
> Hi,
> 
> Thanks for following this up. It is entirely likely that I did something
> wrong and ended up in the wrong place. If so, please bear with me and I
> will try again.
> 
> >Peter, when you did the git bisect, which versions did you end up
> >testing? I ask because this commit later got reverted.
> 
> I bisected between tag v6.5.6 (as good, since it was the last version I
> knew to work fine), and tag v6.5.7 (as bad, which was where I first
> noticed the problem).
> 
> I have copied the bisect log at the bottom of this email. Note that I
> skipped quite a few, since there were other unrelated errors I
> encountered at quite a few commits.
> 
> Strangely, I notice from the log that the revert commit for the one I
> ended up at was one of the ones I ended up skipping. I don't remember
> exactly what the error was though that caused me to skip it.



> I will also try without autorefresh, and see if the same thing happens.
> 
> >I wonder Peter didn't use the last version. If so, I wonder the bisect
> >ended up to a commit unrelated to the correct commit. Erf.
> 
> Anyway, I can do the bisecting again no problem, if I didn't do
> something quite right.
> 
> >The only other thing this patch did was set the rtime correctly, which
> >was later dealt with more carefully in Abdo's patches
> >c094304f937e13e80ce464ff7819affd39d298cc and
> >0c17350e4f72282a7a8ac3180de713b3b322dcea. It might help to know
> >whether the bug is present at that point.
> 
> Should I maybe test these two next?
> 
> Thanks again,
> 
> Peter.

Probably unrelated, but I'm confused about this "unrelated error" that
caused you to skip commits: 115d141218c3deb7d2fc92ea87732b4420c4a5d2,
which you skipped, is one documentation change away from v6.5.7, which
you marked bad.

It would be really great if you could somehow test one of the commits
that happened after this got reverted and before the rtime-fixing
patches got in later, just as a sort of sanity check. The only
candidates for this that you haven't skipped are
3a056519f9773f96dce936be789c913ad27e321c,
03d963913c3281301bd96304cfc6950c5f9c23f9, and
eb90932d4a38b1724aeb1dcbdc0de2ef775f694a.

Otherwise, I don't have any good ideas, so unless Nicholas has some, I
guess we have to dig in and get more details about what's going wrong.
Can you try 25513e90387f60831b94c6041de08db5ba12e95f again with
debugging on (offlineimap -d all) and also with the extra print
statements I've offered below? Also, can you send this info (minus
passwords!) for one or two runs before the one that failed (as well as
for the one that failed).

Thanks,

-- J.M.


---
 offlineimap/accounts.py       | 6 ++++++
 offlineimap/folder/Maildir.py | 4 ++++
 2 files changed, 10 insertions(+)

diff --git a/offlineimap/accounts.py b/offlineimap/accounts.py
index 62ed5c3..f16c65b 100644
--- a/offlineimap/accounts.py
+++ b/offlineimap/accounts.py
@@ -428,6 +428,8 @@ def syncfolder(account, remotefolder, quick):
             statusfolder.deletemessagelist()
 
         statusfolder.cachemessagelist()
+        print "DEBUG: accounts.py statusfolder.cachemessagelist() repos = %s, uids = %s" \
+            % (statusfolder.getname(), statusfolder.getmessageuidlist())
 
         if quick:
             if (not localfolder.quickchanged(statusfolder) and
@@ -440,6 +442,8 @@ def syncfolder(account, remotefolder, quick):
         ui.syncingfolder(remoterepos, remotefolder, localrepos, localfolder)
         ui.loadmessagelist(localrepos, localfolder)
         localfolder.cachemessagelist()
+        print "DEBUG: accounts.py localfolder.cachemessagelist() repos = %s, uids = %s" \
+            % (localfolder.getname(), localfolder.getmessageuidlist())
         ui.messagelistloaded(localrepos, localfolder, localfolder.getmessagecount())
 
         # If either the local or the status folder has messages and
@@ -464,6 +468,8 @@ def syncfolder(account, remotefolder, quick):
         # Load remote folder.
         ui.loadmessagelist(remoterepos, remotefolder)
         remotefolder.cachemessagelist()
+        print "DEBUG: accounts.py remotefolder.cachemessagelist() repos = %s, uids = %s" \
+            % (remotefolder.getname(), remotefolder.getmessageuidlist())
         ui.messagelistloaded(remoterepos, remotefolder,
                              remotefolder.getmessagecount())
 
diff --git a/offlineimap/folder/Maildir.py b/offlineimap/folder/Maildir.py
index 77f7ebb..c10b5bf 100644
--- a/offlineimap/folder/Maildir.py
+++ b/offlineimap/folder/Maildir.py
@@ -345,6 +345,8 @@ class MaildirFolder(BaseFolder):
         if rtime is None:
             rtime = emailutil.get_message_date(content)
         messagename = self.new_message_filename(uid, flags, rtime=rtime)
+        print "DEBUG: Maildir savemessage() uid = %d, rtime = %s" \
+            % (uid, rtime)
         tmpname = self.save_to_tmp_file(messagename, content)
         if rtime != None:
             os.utime(os.path.join(self.getfullname(), tmpname), (rtime, rtime))
@@ -424,6 +426,8 @@ class MaildirFolder(BaseFolder):
         rtime = emailutil.get_message_date(content)
         newfilename = os.path.join(dir_prefix,
           self.new_message_filename(new_uid, flags, rtime=rtime))
+        print "DEBUG: Maildir change_message_uid() uid = %d, new_uid = %d, rtime = %s" \
+                % (uid, new_uid, rtime)
         os.rename(os.path.join(self.getfullname(), oldfilename),
                   os.path.join(self.getfullname(), newfilename))
         self.messagelist[new_uid] = self.messagelist[uid]
-- 
2.4.6