Bug#914034: bug in Net::SSLeay?
Guilhem Moulin
guilhem at debian.org
Sun Apr 7 16:46:13 BST 2019
Control: usertag -1 bsp-2019-04-se-gothenburg
Hi there,
strace(1) shows a select(2) syscall indicating that the socket is ready for
both read and write, but is later blocking on a read(2) without any
write(2) taking place.
select(8, [3], [3], NULL, {tv_sec=180, tv_usec=0}) = 2 (in [3], out [3], left {tv_sec=179, tv_usec=999998})
read(3, "…", 5) = 5
read(3, "…", 156) = 156
read(3,
Net::SSLeay warns:
If you need to select(2) on the socket, go right ahead, but be warned that
OpenSSL does some internal buffering so SSL_read does not always return
data even if the socket selected for reading (just keep on selecting and
trying to read). "Net::SSLeay" is no different from the C language OpenSSL
in this respect.
And indeed LWP::Protocol::http's use of select(2) on SSL sockets *does*
assume that read/write readiness won't block. (If Net::SSLeay::read()
returns -1, then the loop will retry later with SSL_ERROR_WANT_READ/WRITE.)
However since OpenSSL 1.1.1 the SSL_MODE_AUTO_RETRY flag is on by
default, which breaks that assumption: ssl_read(3) might block, even
when select(2) claimed the socket had data to be read.
SSL_MODE_AUTO_RETRY
During normal operations, non-application data records might need to be
sent or received that the application is not aware of. If a
non-application data record was processed, SSL_read_ex(3) and SSL_read(3)
can return with a failure and indicate the need to retry with
SSL_ERROR_WANT_READ. If such a non-application data record was processed,
the flag SSL_MODE_AUTO_RETRY causes it to try to process the next record
instead of returning.
[…]
In a blocking environment, applications are not always prepared to deal
with the functions returning intermediate reports such as retry requests,
and setting the SSL_MODE_AUTO_RETRY flag will cause the functions to only
return after successfully processing an application data record or a
failure.
[…]
All modes are off by default except for SSL_MODE_AUTO_RETRY which is on by
default since 1.1.1.
— https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_mode.html
See also https://github.com/openssl/openssl/issues/6234 .
I see several paths forward here:
- Refactor LWP::Protocol::http's select loop to solve the assumption
that's now broken with OpenSSL ≥1.1.1; or
- Unset SSL_MODE_AUTO_RETRY in IO::Socket::SSL; or
- Make context flags configurable in IO::Socket::SSL, and unset
SSL_MODE_AUTO_RETRY from LWP.
IMHO the first option is not ideal so late in the release cycle. The
second option is the easiest to implement, and should™ be regression-free,
but might confuse people who became used to OpenSSL ≥1.1.1's new context
default flags.
SSL_CTX_clear_mode(3) and SSL_CTRL_CLEAR_MODE macros are unfortunately
not exposed to Net::SSLeay 1.85-2. The proper fix would be to expose
these and release a new version of Net::SSLeay, of course, but for tests
the macros can be taken from /usr/include/openssl/ssl.h:
# define SSL_CTRL_CLEAR_MODE 78
[…]
# define SSL_CTX_clear_mode(ctx,op) \
SSL_CTX_ctrl((ctx),SSL_CTRL_CLEAR_MODE,(op),NULL)
and used as is in IO::Socket::SSL.pm. With the following patch I'm
again able to POST to HTTPS servers using TLS 1.3.
--8<--------------------------------------------------------------->8--
--- a/IO/Socket/SSL.pm
+++ b/IO/Socket/SSL.pm
@@ -2433,6 +2433,7 @@
# cannot guarantee, that the location of the buffer stays constant
Net::SSLeay::CTX_set_mode( $ctx,
SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER|SSL_MODE_ENABLE_PARTIAL_WRITE);
+ Net::SSLeay::CTX_ctrl($ctx, 78, Net::SSLeay::MODE_AUTO_RETRY(), undef);
if ( my $proto_list = $arg_hash->{SSL_npn_protocols} ) {
return IO::Socket::SSL->_internal_error("NPN not supported in Net::SSLeay",9)
--8<--------------------------------------------------------------->8--
(Again, I'm not proposing to patch IO::Socket::SSL as above :-) With
MODE_AUTO_RETRY set — the default for OpenSSL ≥1.1.1 — one gets:
$ strace -e trace=read,write,select perl -MLWP::UserAgent -e 'LWP::UserAgent->new(ssl_opts =>
{SSL_version => "TLSv1_3"})->post("https://facebook.com", { data => "plonc" })';
[…]
select(8, [3], [3], NULL, {tv_sec=180, tv_usec=0}) = 2 (in [3], out [3], left {tv_sec=179, tv_usec=999998})
read(3, "…", 5) = 5
read(3, "…", 156) = 156
read(3,
And now with the MODE_AUTO_RETRY flag unset:
$ select(8, [3], [3], NULL, {tv_sec=180, tv_usec=0}) = 2 (in [3], out [3], left {tv_sec=179, tv_usec=999998})
read(3, "…", 5) = 5
read(3, "…", 156) = 156
write(3, "…", 217) = 217
select(8, [3], NULL, NULL, {tv_sec=180, tv_usec=0}) = 1 (in [3], left {tv_sec=179, tv_usec=870931})
read(3, "…", 5) = 5
read(3, "…", 361) = 361
Cheers,
--
Guilhem.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-perl-maintainers/attachments/20190407/3b35c212/attachment.sig>
More information about the pkg-perl-maintainers
mailing list