Bug#914034: Bug#911938: libhttp-daemon-ssl-perl FTBFS: tests fail: Connection refused

[Guilhem Moulin]

On Tue, 14 May 2019 at 03:57:46 +0200, Steffen Ullrich wrote:
>> Ah I see, thanks for the clarification.  I thought you meant it could
>> yield a deadlock.  Aren't temporary failures also possible on plain
>> sockets (though of course the extra SSL layer make it strictly more
>> likely to happen)?  IIRC if the checksum of the incoming packet
>> mismatches, which causes the read() call to block until the packet is
>> retransmitted.
> 
> select only shows an fd ready  if data are available for read in the socket
> buffer. Data with wrong checksum are discarded by the kernel before they are
> put into the socket buffer and thus don't cause select to show it ready for
> read.
> 
> select(2) explicitly says:
> 
> A file descriptor is considered ready if it is possible to perform a
> corresponding I/O operation (e.g., read(2)  without blocking ...

Hmm, however in the “Bugs” sections, it says it's in fact not the case, and
that non-blocking I/O should be used to avoid temporary failures:

    Under Linux, select() may report a socket file descriptor as "ready for
    reading", while nevertheless a subsequent read blocks.  This could for
    example happen when data has arrived but upon examination has wrong
    checksum and is discarded.  There may be other circumstances in which a
    file descriptor is spuriously reported as ready.  Thus it may be safer to
    use O_NONBLOCK on sockets that should not block.
 
>>> And while using blocking I/O with polling sockets might be fine with plain
>>> sockets where each byte is part of application data it is not fine with SSL
>>> since the unit in SSL is not a byte but a record and some records might
>>> contain application data and some not.
>>
>> Why is that not fine, if the SSL_read() caller is ready for that (documented)
>> outcome, and doesn't assume that the call will always block until some
>> application data is received?
> 
> IO::Socket::SSL is intended as abstraction which behaves as much as possible
> as other IO::Socket classes. It is not intended that the developer has to be
> familiar with the exact semantics of SSL_read (which also changed over
> time, especially with OpenSSL 1.1.0 and again with OpenSSL 1.1.1). While it
> is impossible to behave in exactly all cases sysread is usually not expect
> to return a temporary error on a blocking socket.
> […]
> Yes, the intention was to reflect as much as possible what is expected from
> IO::Socket::sysread and not what the SSL_read documentation says.

Fair enough.  Then it sounds like you'd want to set SSL_MODE_AUTO_RETRY
explicitly and not rely on the OpenSSL old or new defaults :-)  (Or loop in
Perl if support for OpenSSL that are two decades old is desired.)

For what it's worth, I interpreted

    Also, calls to sysread might fail, because it must first finish an SSL
    handshake.
    To understand these behaviors is essential, if you write applications
    which use event loops and/or non-blocking sockets.

from the sysread() documentation as an invitation to read the low-level
documentation and see what SSL_read() may return, also with blocking I/O :-)
(After all since sysread() will never block if there is some unprocessed data
left in the current SSL frame, that's already a hint that this sysread() has
some peculiarities that are not found in the version with plain sockets.)  The
new documentation clarifies a bit the expectation, thanks!  But I guess it
would be clearer if the paragraphs I quoted above were explicitly said
to only apply to non-blocking I/O.

Also isn't the workaround you implemented earlier

    “deal with OpenSSL 1.1.1 switching on SSL_AUTO_RETRY by default by disabling it when non-blocking”
    https://github.com/noxxi/p5-io-socket-ssl/commit/09bc6a3203bc7bc89078317da42a3e96cdbf94fc

a no-op?  AFAICT setting SSL_MODE_AUTO_RETRY avoids SSL_read() returning
SSL_ERROR_WANT_READ when a non-application data record is received, and
instead makes it block until application data is received.  For non-blocking
I/O however, SSL_read() will of course never block and may — regardless of
whether SSL_MODE_AUTO_RETRY is set — return SSL_ERROR_WANT_{READ,WRITE}.

-- 
Guilhem.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-perl-maintainers/attachments/20190514/84c2e597/attachment.sig>