Bug#912206: freerdp2-x11: ERRCONNECT_TLS_CONNECT_FAILED with libssl1.1 1.1.1-1

[Kevin Locke]

Package: freerdp2-x11
Version: 2.0.0~git20180411.1.7a7b1802+dfsg1-2+b1
Severity: normal

Dear Maintainer,

After upgrading libssl1.1 from 1.1.0h-4 to 1.1.1-1 xfreerdp is no longer
able to connect to a computer running Remote Desktop Services on Windows
Server 2008 R2 (with default settings as far as I am aware) using TLS
security.  Connection fails with the following messages:

    [ERROR][com.freerdp.core] - freerdp_set_last_error ERRCONNECT_TLS_CONNECT_FAILED [0x00020008]
    [ERROR][com.freerdp.core.connection] - Error: protocol security negotiation or connection failure

Downgrading libssl1.1 to 1.1.0h-4 fixes the issue.  To further diagnose
the cause, I noticed that the server sends TCP RST in response to the
SSL Client Hello message.  After some trial and error, I determined that
this occurs whenever rsa_pkcs1_sha1 in not the offered signature
algorithms, which is the case for SECLEVEL=2 which is the default in the
libssl1.1 Debian package since version 1.1.1~~pre6-1.  To confirm, this
fails:

    openssl s_client -connect 192.168.0.2:3389

while this works:

    openssl s_client -cipher DEFAULT at SECLEVEL=1 -connect 192.168.0.2:3389

For further confirmation that rsa_pkcs1_sha1 is responsible, this works:

    openssl s_client -cipher DEFAULT at SECLEVEL=1 -sigalgs rsa_pkcs1_sha1 -connect 192.168.0.2:3389

while this fails:

    openssl s_client -cipher DEFAULT at SECLEVEL=1 -sigalgs RSA+SHA512:DSA+SHA512:ECDSA+SHA512:RSA+SHA384:DSA+SHA384:ECDSA+SHA384:RSA+SHA256:DSA+SHA256:ECDSA+SHA256:RSA+SHA224:DSA+SHA224:ECDSA+SHA224:DSA+SHA1:ECDSA+SHA1 -connect 192.168.0.2:3389

Applying this discovery, it is possible to make xfreerdp work using:

    xfreerdp /tls-ciphers:DEFAULT at SECLEVEL=1

However, since most users are unlikely to figure this out on their own,
I'd suggest calling SSL_CTX_set_security_level to set the security level
to 1 or improving the error message to suggest this workaround.

Thanks,
Kevin


-- System Information:
Debian Release: buster/sid
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable'), (101, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.19.0 (SMP w/4 CPU cores)
Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8), LANGUAGE=en_US.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages freerdp2-x11 depends on:
ii  libc6                 2.27-6
ii  libfreerdp-client2-2  2.0.0~git20180411.1.7a7b1802+dfsg1-2+b1
ii  libfreerdp2-2         2.0.0~git20180411.1.7a7b1802+dfsg1-2+b1
ii  libwinpr2-2           2.0.0~git20180411.1.7a7b1802+dfsg1-2+b1
ii  libx11-6              2:1.6.7-1
ii  libxcursor1           1:1.1.15-1
ii  libxext6              2:1.3.3-1+b2
ii  libxfixes3            1:5.0.3-1
ii  libxi6                2:1.7.9-1
ii  libxinerama1          2:1.1.4-1
ii  libxrandr2            2:1.5.1-1
ii  libxrender1           1:0.9.10-1
ii  libxv1                2:1.0.11-1

freerdp2-x11 recommends no packages.

freerdp2-x11 suggests no packages.

-- no debconf information