[PATCH] fix hang because of infinite loop reading EOF

[Haojun Bao]

Hi, all

Offlineimap sometimes hangs when downloading gmail with large
attachment.

This is because of several infinite loop when reading the socket while
the EOF has happened. It can be seen from 2 facts:

    1. strace will show read(2) syscall repeatedly returning 0 on the
    same fd.

    2. netstat will show the socket in CLOSE_WAIT state. 

Also, when the hang happens, if you don't kill offlineimap, the system
memory will be used up by it in the end, thus causing a system hang. Why
this is so I didn't get it quitely. (BTW, the CPU is 100% as soon as the
read(2) infinite loop begin).

After reading the mailing list, I see Gábor Melis has proposed a patch
(SO_KEEPALIVE) to fix hang, I don't know if my case is same as his. But
anyway, I think the several read methods contain bugs in their
implementation.

You can use the following patch to fix the problem.

(I have posted it on the news server, but not sure did it get through.)

Signed-off-by: Bao Haojun <baohaojun at gmail.com>

---
 offlineimap/imaplibutil.py |    9 ++++++++-
 offlineimap/imapserver.py  |    4 ++++
 2 files changed, 12 insertions(+), 1 deletions(-)

diff --git a/offlineimap/imaplibutil.py b/offlineimap/imaplibutil.py
index 69d3983..ad98ab8 100644
--- a/offlineimap/imaplibutil.py
+++ b/offlineimap/imaplibutil.py
@@ -47,7 +47,10 @@ class IMAP4_Tunnel(IMAP4):
     def read(self, size):
         retval = ''
         while len(retval) < size:
-            retval += self.infd.read(size - len(retval))
+            buf = self.infd.read(size - len(retval))
+            if not buf:
+                break
+            retval += buf
         return retval
 
     def readline(self):
@@ -200,6 +203,8 @@ class WrappedIMAP4_SSL(IMAP4_SSL):
         read = 0
         while read < n:
             data = self._read_upto (n-read)
+            if not data:
+                break
             read += len(data)
             chunks.append(data)
 
@@ -212,6 +217,8 @@ class WrappedIMAP4_SSL(IMAP4_SSL):
         retval = ''
         while 1:
             linebuf = self._read_upto(1024)
+            if not linebuf:
+                return retval
             nlindex = linebuf.find("\n")
             if nlindex != -1:
                 retval += linebuf[:nlindex + 1]
diff --git a/offlineimap/imapserver.py b/offlineimap/imapserver.py
index 2a9f247..c298aa3 100644
--- a/offlineimap/imapserver.py
+++ b/offlineimap/imapserver.py
@@ -71,6 +71,8 @@ class UsefulIMAP4(UsefulIMAPMixIn, imaplibutil.WrappedIMAP4):
             io = StringIO()
             while read < size:
                 data = imaplib.IMAP4.read (self, min(size-read,8192))
+                if not data:
+                    break
                 read += len(data)
                 io.write(data)
             return io.getvalue()
@@ -86,6 +88,8 @@ class UsefulIMAP4_SSL(UsefulIMAPMixIn, imaplibutil.WrappedIMAP4_SSL):
             io = StringIO()
             while read < size:
                 data = imaplibutil.WrappedIMAP4_SSL.read (self, min(size-read,8192))
+                if not data:
+                    break
                 read += len(data)
                 io.write(data)
             return io.getvalue()
-- 
1.7.2.3