maxage causes loss of local email

[Janna Martl]

On Sat, Mar 21, 2015 at 12:49:35PM +0100, Nicolas Sebrecht wrote:
>On Sat, Mar 21, 2015 at 02:14:08AM -0400, Janna Martl wrote:
>
>> I realized where (some of) my date-mismatched messages were coming from:
>> if you increase maxage, then some local messages get re-copied to
>> remote (i.e. the ones from ~ maxage days ago), they're assigned (new)
>> UIDs. This is bad because, a day or so later, these messages are in
>> the remote messagelist (new UID) but not in the local messagelist (old
>> date), so they get deleted locally.
                              ^remotely

>In the worse case this should be documented but this should not happen
>anymore with the latter fix about timezones.

No, it does still happen; it's not caused by timezone issues, but
rather by the way the timezone fix works. The problem is basically
that the remote messagelist contains all messages with UID > min_uid,
but the local messagelist contains messages that have UID > min_uid
*and also* are newer than maxage. In the above case, we've got a
message that fulfills the first, but not second, condition.

Possible fix #1: Ideally, the local messagelist should also contain
simply messages with UID > min_uid. For reference, in
folder/Maildir.py:

def _scanfolder(self):
	...
	for dirannex in ['new', 'cur']:
		fulldirname = os.path.join(self.getfullname(), dirannex)
		files.extend((dirannex, filename) for
					 filename in os.listdir(fulldirname))

	for dirannex, filename in files:
	...

Unfortunately, os.listdir() gives filenames in a random order. So I
could do something like:
	for dirannex, filename in files:
		if filename is newer than maxage:
			add it to newlist
		else:
			add it to oldlist
	min_uid = min(uid's in newlist)
	for filename in oldlist:
		if filename's uid is >= min_uid
			add it to newlist
This requires going through a long list twice.

Possible fix #2: when the offending message gets copied to the remote,
it gets assigned a new UID, and then the local message UID (and
filename) gets changed in change_message_uid(). We could make the time
in the new filename = the current time. (Basically, we want the
filename time to be not the internal time, or the download time, but
rather the time when the UID was assigned.) This seems a little ad hoc
but is more efficient than #1.

Opinions?

>I have some little comments but I'm not asking for another round; I'll
>fix them myself if they are fine for you.

Makes sense; thanks. (but maybe wait until we've resolved the above
issue too?)

-- J.M.