Bug#945507: systemd-resolved rejects DNS-over-TLS based on GNUTLS_E_RECEIVED_ILLEGAL_PARAMETER even though gnutls-cli works fine

Tue Nov 26 02:18:02 GMT 2019

Package: systemd
Version: 243-8

On an amd64 system running sid,

with the following settings reported by resolvectl:

  DNSOverTLS setting: opportunistic
      DNSSEC setting: allow-downgrade
    DNSSEC supported: no
  Current DNS Server: 199.58.81.218
         DNS Servers: 199.58.81.218
                      2001:470:1c:76d::53

The TLS connections don't work for some reason (the host above is
dns.cmrg.net, which only offers DNS-over-TLS).

/etc/resolv.conf is a symlink to /lib/systemd/resolv.conf

I attached ltrace to the systemd-resolved process, while trying to
elicit a domain name with "ping" and saw this interaction with GnuTLS:

3437 20:49:39 gnutls_init(0x7ffcfe82eae8, 266, 16, 608)                                     = 0
3437 20:49:39 gnutls_priority_set_direct(0x55f47918c140, 0x55f4772712b8, 0, 0)              = 0
3437 20:49:39 gnutls_credentials_set(0x55f47918c140, 1, 0x55f478eb0140, 0)                  = 0
3437 20:49:39 gnutls_handshake_set_timeout(0x55f47918c140, 0xffffffff, 0, 0x55f47918a930)   = 0x9c40
3437 20:49:39 gnutls_transport_set_ptr2(0x55f47918c140, 19, 0x55f47918a680, 0x55f47918a930) = 0x9c40
3437 20:49:39 gnutls_transport_set_vec_push_function(0x55f47918c140, 0x55f47723cc80, 0x55f47918a680, 0x55f47918a930) = 0x9c40
3437 20:49:39 gnutls_handshake(0x55f47918c140, 0x55f47723cc80, 0x55f47918a680, 0x55f47918a930 <unfinished ...>
3437 20:49:39 sendmsg(19, 0x7ffcfe82e630, 0x20000000, 1)                                    = -1
3437 20:49:39 __errno_location()                                                            = 0x7fc6d84bbac0
3437 20:49:39 __errno_location()                                                            = 0x7fc6d84bbac0
3437 20:49:39 <... gnutls_handshake resumed> )                                              = 0xffffffe4
3437 20:49:39 gnutls_error_is_fatal(0xffffffe4, 0, -128, 0)                                 = 0

0xffffffe4 is -55, which is GNUTLS_E_RECEIVED_ILLEGAL_PARAMETER,
according to:

https://gnutls.org/manual/gnutls.html#Error-codes

I'm attaching a pcap for all the traffic on port 853 from the above
attempt.  I don't see any obviously illegal parameters there.

In further debugging, i tried using gnutls-cli to connect directly to
it, and that worked fine:

$ gnutls-cli --sni-hostname=dns.cmrg.net --verify-hostname=dns.cmrg.net 199.58.81.218:853
Processed 128 CA certificate(s).
Resolving '199.58.81.218:853'...
Connecting to '199.58.81.218:853'...
- Certificate type: X.509
- Got a certificate list of 2 certificates.
- Certificate[0] info:
 - subject `CN=dns.cmrg.net', issuer `CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US', serial 0x03a4d7448cc89c9444776bbf992fe74a4252, RSA key 2048 bits, signed using RSA-SHA256, activated `2019-11-01 06:00:16 UTC', expires `2020-01-30 06:00:16 UTC', pin-sha256="3IOHSS48KOc/zlkKGtI46a9TY9PPKDVGhE3W2ZS4JZo="
	Public Key ID:
		sha1:44be3735f2f6cf668b6143335d8189250a7c5cd3
		sha256:dc8387492e3c28e73fce590a1ad238e9af5363d3cf283546844dd6d994b8259a
	Public Key PIN:
		pin-sha256:3IOHSS48KOc/zlkKGtI46a9TY9PPKDVGhE3W2ZS4JZo=

- Certificate[1] info:
 - subject `CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US', issuer `CN=DST Root CA X3,O=Digital Signature Trust Co.', serial 0x0a0141420000015385736a0b85eca708, RSA key 2048 bits, signed using RSA-SHA256, activated `2016-03-17 16:40:46 UTC', expires `2021-03-17 16:40:46 UTC', pin-sha256="YLh1dUR9y6Kja30RrAn7JKnbQG/uEtLMkBgFF2Fuihg="
- Status: The certificate is trusted. 
- Description: (TLS1.3)-(ECDHE-SECP256R1)-(RSA-PSS-RSAE-SHA256)-(AES-256-GCM)
- Options:
- Handshake was completed

- Simple Client Mode:

I'm attaching a pcap from the gnutls-cli connection as well.

Note from the pcaps that the gnutls-cli connection manages to negotiate
TLS 1.3, while the systemd-resolved connection only manages to elicit a
TLS 1.2 response from the server for some reason.

I'm seeing this error in systemd-resolved with libgnutls30 3.6.10-5, but
I also tried this while rolling back to older versions of libgnutls30 --
version 3.6.7-4 from buster, for example -- and it didn't fix the
problem.

So i think the issue is something to do with the way that libgnutls is
being initialized in this version of systemd.

I do not see this misbehavior on a comparable VM running debian buster
(with systemd 241-7~deb10u2).  on the buster VM, the nameservice
works fine with systemd-resolved.

let me know if you want me to try some other debugging step.

    --dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: systemd-resolved.pcapng
Type: application/octet-stream
Size: 5068 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-systemd-maintainers/attachments/20191125/6ead79ad/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gnutls-cli.pcapng
Type: application/octet-stream
Size: 6220 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-systemd-maintainers/attachments/20191125/6ead79ad/attachment-0003.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-systemd-maintainers/attachments/20191125/6ead79ad/attachment-0001.sig>