Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

[Guilhem Moulin]

On Mon, 13 May 2019 at 22:24:55 +0200, Steffen Ullrich wrote:
> On Mon, May 13, 2019 at 03:18:14PM +0200, Guilhem Moulin <guilhem at debian.org> wrote:
>> Uh, what?  “Before” meaning with ≤TLSv1.2, or with OpenSSL <1.1.1a's
>> default flags?  libssl mentions no such thing beside the new default
>> mode.  And in fact the s_client() program, *from OpenSSL upstream
>> itself*, does precisely that: a select loop in blocking I/O mode (unless
>> the ‘-nbio’ flag is set).
> 
> "Before" meaning already with previous OpenSSL versions. But the effect was
> likely hard to notice: an incomplete SSL record would not cause a permanent
> hang but only a blocking read until the rest of the data was received, i.e.
> only some slow down in some situations.

Ah I see, thanks for the clarification.  I thought you meant it could
yield a deadlock.  Aren't temporary failures also possible on plain
sockets (though of course the extra SSL layer make it strictly more
likely to happen)?  IIRC if the checksum of the incoming packet
mismatches, which causes the read() call to block until the packet is
retransmitted.

> And while using blocking I/O with polling sockets might be fine with plain
> sockets where each byte is part of application data it is not fine with SSL
> since the unit in SSL is not a byte but a record and some records might
> contain application data and some not.

Why is that not fine, if the SSL_read() caller is ready for that (documented)
outcome, and doesn't assume that the call will always block until some
application data is received?

>> The documentation (libssl's SSL_read()'s [0], Net::SSLeay::read() [1],
>> as well as IO::Socket::SSL::sysread() [2]) all warn that reading from an
>> SSL socket has a different behavior than usual read(2) system calls, in
>> that read failures should be treated with care, as there are both
>> retryable (eg. when no application data was received) and non-retryable
>> errors.
> 
> As I was the one who wrote the documentation for IO::Socket::SSL: the part
> about sysread failing was supposed to be about non-blocking sockets. A
> blocking socket was not expected to return nothing just because the
> handshake was not done.

Then I don't get it.  AFAICT the current documentation reflects what
happens with blocking sockets and OpenSSL <1.1.1a, namely that
IO::Socket::SSL::sysread() returns undef when the TLS ≤1.2 session is
renegotiated.  Are you saying that the intention was to keep retrying
(either Perl-side, or within SSL_read() via SSL_MODE_AUTO_RETRY with
OpenSSL ≥0.9.6) until application data is received or a fatal error is
encountered?
 
> Do you know a relevant module or actually used application which has
> problems because of the default mode of SSL_MODE_AUTO_RETRY the ability to
> change it would fix the related problems and not introduce new ones, i.e.
> where such a fix would be the right thing to do?

Not anything using IO::Socket::SSL, I'm afraid.  I'm personally more
used to the lower-level APIs like libssl and Net::SSLeay.  For libssl in
particular, a quick codesearch returns a few prominent programs that
reverted the new OpenSSL default.

  Apache2's mod_ssl:
    Bug:    https://github.com/openssl/openssl/issues/7178
    Commit: https://github.com/apache/httpd/commit/6ee9d597e01281f2ef2e146586129af6aed7854d#diff-b70bd458eb699e70c322ee797a3e0991

  Neon:
    File:   http://svn.webdav.org/repos/projects/neon/tags/0.30.2/src/ne_socket.c
    Change: http://lists.manyfish.co.uk/pipermail/neon-commits/2018-September/001077.html
  
  YottaDB
    Bug: https://github.com/YottaDB/YDB/commit/1d0a8943d8be24a6f3dd3e4d2c8bfaad1fb75c87

> Another option might be to have an option SSL_mode_auto_retry which by
> default is not set but might be set to either 0 or 1 to get a consistent
> behavior across different OpenSSL versions.

I quite like this approach :-)

Anyway this bug can probably be closed now, maybe we should follow up
elsewhere.  Thanks for the discussion & analysis!
-- 
Guilhem.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-perl-maintainers/attachments/20190514/3e309d26/attachment.sig>