Expected semantics of LocalStatus files

[Rob Browning]

Hi,

I've been evaluating offlineimap lately, and I noticed that it was very
slow when dealing with large maildirs.  After some investigation, I
tracked the problem down to the code in LocalStatus.py that writes the
status file.

The problem is that LocalStatus.py rewrites the entire file for every
delivery, and once that file is over 1-2MB, offlineimap keeps the CPU
pegged, and only delivers a message or two per second.

As a test, I set doautosave to 0 (in a test dir), and the performance
shot back up to 30+ messages/sec with much lower CPU use.

That made me wonder about the semantics of the LocalStatus files.
i.e. what happens if they're missing?  Is that a catastrophic failure,
or can they be re-generated?  If the latter, then I thought it might be
possible to improve the performance by just deleting the file at the
beginning of a sequence of copies, and then rewrite it just once
(atomically) after all the updates for that folder have been finished.
Any intervening crash would just leave the file missing.

Thanks
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4