hanging connections

[Gábor Melis]

Sebastian Spaeth <Sebastian at SSpaeth.de> writes:

> On Fri, 25 Feb 2011 22:23:07 +0100, mega at retes.hu wrote:
>> In the end it seems to be cured by:
>> +        self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
>
> Hi there. I am glad you fixed the hangs for you. I am missing a bit of
> context for this patch though. Why do you believe can this patch fix the
> hangs?

Well, because I tested it :-).

The long story is that I configured the system level keepalive
parameters so that my ssh (client) connections stay alive and my cisco
router-modem does not kill 'inactive' tcp connections. For ssh that was
enough because, I think, it sets SO_KEEPALIVE.

Anyway, setting the system keepalive parameters had no effect on
offlineimap because one has to explicitly request it by setsockopt
SO_KEEPALIVE.

> Do you believe that gmail might close ssl connections when we
> don't send keep-alive messages and our thread pool gets confused,
> believing they are open and waiting for data when they have been closed
> really?

I have two remote IMAP servers with which I sync. Gmail always worked
fine simply because (I guess) it responds much more quickly and reliably
then the IMAP server at work that's on the other side of the globe.

So it's not Gmail, but that's kind of unimportant. The important thing
is that I checked on the local machine and the server and netstat showed
an ESTABLISHED connection on both while the connection was dropped
[probably] by my router a long time ago and offlineimap just sit there
waiting.

> Why would they, do we keep idle ssl connections around that are being
> shut down?
>
> If something like this is the case, then I believe we should improve on
> 2 fronts: a) send keep-alive messages and b) make us detect dead
> connections better and c) change our timeout strategies so we don't get
> them closed on us in the first place.
>
> I am not a networking specialist. man 7 socket says:
>        SO_KEEPALIVE
>               Enable sending of  keep-alive  messages  on  connection-oriented
>               sockets.  Expects an integer boolean flag.
>
> This sounds like a good thing to me, which wouldn't harm in any case. So
> I am all for this patch and will send a properly formatted patch as a
> reply to this one, containing a Signed-off-by from you, which is
> hopefully ok for you.
>
> Nicolas, this patch can't hurt and might fix gmail hangs. Is it a
> candidate for the stable (and next) branch? We should still not deadlock
> if ssl closes connections on us without raising Exceptions though, so
> this requires further thought.

One could argue that socktimeout covers this use-case, but IMHO they are
complementary.

> Sebastian
> _______________________________________________
> OfflineIMAP-project mailing list
> OfflineIMAP-project at lists.alioth.debian.org
> http://lists.alioth.debian.org/mailman/listinfo/offlineimap-project
>
> OfflineIMAP homepage: http://software.complete.org/offlineimap