<DKIM> maxage causes loss of local email

[Janna Martl]

There's more to this than I thought at first.

On Tue, Mar 03, 2015 at 12:16:47PM +0100, Nicolas Sebrecht wrote:
>So, in folder/Maildir.py we set the timestamp with
>
>  def savemessage(self, uid, content, flags, rtime):
>      <...>
>      if rtime != None:
>          os.utime(os.path.join(self.getfullname(), tmpname), (rtime, rtime))
>      <...>

At this point, for me, rtime = None, so this isn't doing anything.
This function is called by copymessageto() in Base.py, and the rtime
it passes to savemessage() is set in the following lines:

            rtime = self.getmessagetime(uid)
            if dstfolder.utime_from_message:
                content = self.getmessage(uid)
                rtime = emailutil.get_message_date(content, 'Date')

I have two things to say about these lines:

(1) In line 1, self.getmessagetime() (here self = the remote folder) returns
None for me. This is because in:

    def cachemessagelist(self):
		...
		rtime = imaplibutil.Internaldate2epoch(messagestr)
		self.messagelist[uid] = {'uid': uid, 'flags': flags, 'time': rtime}

times seem to be gotten by parsing messagestr, but my messagestr's
look like (UID 45437 FLAGS ()) and don't contain anything resembling a
date.

(2) In lines 2-4 of the above, utime_from_message is False. This
seems to be an undocumented configuration option (introduced in commit
3bc66c0858ec) whose default option is False. If this is set to True
instead, then lines 3 and 4 actually happen: now rtime is the
internal date (which is not changed by the later call to os.utime),
but the filename still has the download date.

This is bad, because Maildir.py's _iswithinmaxage() still checks the
filename to determine the age of a local message.

>  def _scanfolder(self):
>      <...>
>      for dirannex, filename in files:
>          <...>
>          if maxage and not self._iswithinmaxage(filename, maxage):
>              continue
>          <...>

>Janna, could you confirm that maxage is honored for you and that
>_scanfolder() excludes mails from your Maildir?

So: if I set utime_from_message = True, then the message ends up on the
local messagelist with the correct time, during the run in which it is
downloaded. But if offlineimap is run again later, it uses
_scanfolder() to reconstruct the local messagelist, and the only
maxage-related exclusion is done via _iswithinmaxage(), which only
looks at filenames.


>I wonder if the timestamps are actually written to the filesystem.
>Perhaps it's mounted with noatime?

It is.

-- J.M.