<span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">Applied in my next branch, if no one objects, I'll apply to the official repo next branch tomorrow.</span><br>
<br><div class="gmail_quote">On Mon, Feb 4, 2013 at 5:52 AM, Austin Clements <span dir="ltr"><<a href="mailto:amdragon@mit.edu" target="_blank">amdragon@mit.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Ping.<br>
<br>
I've been running with this patch for some time now and haven't received<br>
any more corrupted email deliveries (though my logs show many instances<br>
where I would have received a corrupted email with the unpatched code).<br>
<br>
On Sat, 29 Dec 2012, Austin Clements <<a href="mailto:amdragon@MIT.EDU">amdragon@MIT.EDU</a>> wrote:<br>
> IMAPFolder.getmessage incorrectly assumes that the 'data' returned by<br>
> imapobj.uid('fetch', ...) will always be of the form<br>
><br>
> [(fetch-info, message-body)]<br>
><br>
> However, this call returns a list of *all* pending untagged FETCH<br>
> responses, not just the requested FETCH. If any message flags were<br>
> changed in the selected IMAP folder since the last command (by another<br>
> client or another thread in OfflineIMAP itself), the IMAP server will<br>
> issue unsolicited FETCH responses indicating these flag changes<br>
> (RFC3501, section 7). When this happens, 'data' will look like, for<br>
> example<br>
><br>
> ['1231 (FLAGS (\\Seen) UID 5300)',<br>
> '1238 (FLAGS (\\Seen) UID 5318)',<br>
> ('1242 (UID 5325 BODY[] {7976}', message-body)]<br>
><br>
> Unfortunately, getmessage retrieves the message body as data[0][1],<br>
> which in this example is just the string "2". This is what ultimately<br>
> gets stored in the maildir file, resulting in a message consisting<br>
> exclusively of a single ASCII digit.<br>
><br>
> Ideally, either imaplib2 or getmessage would parse the fetch responses<br>
> to find the response for the requested UID. However, since IMAP only<br>
> specifies unilateral FETCH responses for flag changes, the body fetch<br>
> will be the only tuple in the response; hence this patch simply finds<br>
> the element of 'data' that is a tuple (aborting if there is more than<br>
> one tuple) and uses that.<br>
><br>
> Multi-threaded OfflineIMAP with IDLE or holdconnectionopen is<br>
> particularly susceptible to this problem because flag changes synced<br>
> back to the IMAP server on one thread will appear as unsolicited FETCH<br>
> responses on another thread if it happens to have the same folder<br>
> selected. This can also happen without IDLE or holdconnectionopen or<br>
> even in single-threaded OfflineIMAP with concurrent access from other<br>
> IMAP clients (webmail clients, etc.), though the window for the bug is<br>
> much smaller.<br>
><br>
> Signed-off-by: Austin Clements <<a href="mailto:amdragon@mit.edu">amdragon@mit.edu</a>><br>
> ---<br>
><br>
> I posted a preliminary version of this patch on December 7th. I<br>
> tweaked it slightly in the following days and have been running this<br>
> continuously since then without any problems.<br>
><br>
> Since the context of this patch has changed dramatically between maint<br>
> and master, this is based on master. maint also has this bug,<br>
> however, and if people want I can write a separate backported patch.<br>
><br>
> offlineimap/folder/IMAP.py | 10 +++++++++-<br>
> 1 file changed, 9 insertions(+), 1 deletion(-)<br>
><br>
> diff --git a/offlineimap/folder/IMAP.py b/offlineimap/folder/IMAP.py<br>
> index 75520c2..6b0ed04 100644<br>
> --- a/offlineimap/folder/IMAP.py<br>
> +++ b/offlineimap/folder/IMAP.py<br>
> @@ -226,7 +226,15 @@ class IMAPFolder(BaseFolder):<br>
> fails_left -= 1<br>
> if not fails_left:<br>
> raise e<br>
> - if data == [None] or res_type != 'OK':<br>
> + # Remove unsolicited FETCH responses caused by flag<br>
> + # changes from concurrent connections. These appear as<br>
> + # strings in 'data', while the BODY response appears as a<br>
> + # tuple. The closing ')' on the BODY response also<br>
> + # appears as a stray string, which this also removes.<br>
> + # This should leave exactly one response in 'data'.<br>
> + if res_type == 'OK':<br>
> + data = [res for res in data if not isinstance(res, str)]<br>
> + if data == [None] or res_type != 'OK' or len(data) != 1:<br>
> #IMAP server says bad request or UID does not exist<br>
> severity = OfflineImapError.ERROR.MESSAGE<br>
> reason = "IMAP server '%s' failed to fetch message UID '%d'."\<br>
> --<br>
> 1.7.10.4<br>
<br>
_______________________________________________<br>
OfflineIMAP-project mailing list<br>
<a href="mailto:OfflineIMAP-project@lists.alioth.debian.org">OfflineIMAP-project@lists.alioth.debian.org</a><br>
<a href="http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/offlineimap-project" target="_blank">http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/offlineimap-project</a><br>
<br>
OfflineIMAP homepage: <a href="http://software.complete.org/offlineimap" target="_blank">http://software.complete.org/offlineimap</a><br>
</blockquote></div><br>