[PATCH] Re: IMAP: improve quickchanged() performance

[Sebastian Spaeth]

On Thu, 14 Apr 2011 19:46:58 +0200, Nicolas Sebrecht wrote:
> On Tue, Apr 12, 2011 at 07:32:11PM +0200, Sebastian Spaeth wrote:
> > On Tue, 12 Apr 2011 19:17:06 +0200, Nicolas Sebrecht <nicolas.s-dev at laposte.net> wrote:
> > > Not sure. Are we talking about _remote_ changes while syncing?
> > 
> > Yes, the quickchanged() checks if it should perform a regular sync or
> > not and it looks at changes on REMOTE.
> 
> So, come back to my original question:
> 
> > > Can't we try to fetch a deleted UID, then? How do we handle such
> > > case?
> IOW if we don't check for changes on remote while syncing, I wonder if
> we won't do insane things like trying to fetch a mail which was just
> deleted (and possibly crash).

Ahh, _now_ I get your issue, sorry I was a bit thick. You are asking what
happens when changes occur on REMOTE while we are syncing.

That is a valid question, and one that we need to investigate
deeper. But it is one that is not at all related to the quick setting or
the quickchanged() function. :-)


It is not a 'quick' issue, because we query the IMAP folder for the
number of mails it has (and the highest UID) and if that is different
From our LocalFolder status, we will just perform a plain old regular
folder sync (which will fetch all IMAP mails and UIDs).

So, this is not an issue with 'quick' syncs. But it is an issue that we
need to investigate for the regular folder sync case. I believe that
when we e.g. try to copy a message that has been deleted since we
fetched the status, we will raise an exception, and the mesage copying
will be aborted. (However, I am not sure, *how* far the Exeption would
bubble, e.g. if it would abort the whole folder sync.)

> The good question is: do we handle the case nicely?

I'd say this calls for a code audit, with this specific issue in mind.

What I think would make sense, is if we introduced an custom exception,
e.g. an OfflineImap.Error(). This exception would have a "reason" string
that we can log and print, it would have an "errno", so we can identify
what went wrong, and it would have a "abort_level" (or similar), that
would indicate at which level we should fail.

e.g. we could have:
abort_level = MESSAGE:
   Just abort this single message sync but continue with the rest of the
   folder. This would e.g. be issued when a single message has been
   found faulty when uploading (e.g. too big), or when a message that we
   want to copy is missing. OfflineImap would not abort when it comes
   across something like this.

abort_level = FOLDER:
   Abort the folder sync, but continue with the rest of the folders in
   this account. This would be raised if e.g. the folder is read-only,
   or cannot be found on the server (e.g. as it has been deleted)

abort_level = ACCOUNT:
   Abort the whole account syncing. E.g. when the mail server is down,
   the password wrong, or the network has some error. If this error
   occurs, we would still try to sync the other accounts, before exiting
   with an error message.

Whenever such an exception would be raise, we store it in a stack, and
on program exit, we output all exceptions that had been queued, so the
user can see what has gone wrong during a sync and can react
accordingly.

This is how I think we could deal sensibly with error condidtions, but
others might have be better ideas, so I am open for feedback.

Sebastian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/offlineimap-project/attachments/20110415/e7972d63/attachment.pgp>