[Pkg-openssl-devel] Sendmail TLS-issue with libssl 1.1.1b-2 - unable to STARTTLS to some MTAs

[Joerg Hinz]

Hello,

I got now 2 weeks stuck with TLS-problems and just found out the problem.

Running Debian SID with libssl1.1_1.1.1b causes sendmail to fail with 
the TLS-handshake ON SOME SPECIFIC hosts:


# Use libssl with issue:

# apt-get install libssl1.1
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
   libssl1.1
1 upgraded, 0 newly installed, 0 to remove and 269 not upgraded.
Need to get 0 B/1525 kB of archives.
After this operation, 545 kB of additional disk space will be used.
Preconfiguring packages ...
(Reading database ... 70524 files and directories currently installed.)
Preparing to unpack .../libssl1.1_1.1.1b-2_i386.deb ...
Unpacking libssl1.1:i386 (1.1.1b-2) over (1.1.0j-1~deb9u1) ...
Processing triggers for libc-bin (2.28-10) ...
Setting up libssl1.1:i386 (1.1.1b-2) ...
Processing triggers for libc-bin (2.28-10) ...


Send a mail to a recipient using messagelabs-Server:

May 21 14:03:11 lindsey sm-mta[9247]: STARTTLS=client, error: connect 
failed=-1, reason=dh key too small, SSL_error=1, errno=0, retry=-1
May 21 14:03:11 lindsey sm-mta[9247]: ruleset=tls_server, arg1=SOFTWARE, 
relay=cluster8.eu.messagelabs.com, reject=403 4.7.0 TLS handshake failed.
May 21 14:03:11 lindsey sm-mta[9247]: x4LC3Amd009245: to=<xxxx.de>, 
delay=00:00:01, xdelay=00:00:01, mailer=esmtp, pri=121212, 
relay=cluster8a.eu.messagelabs.com. [18.194.106.207], dsn=4.0.0, 
stat=Deferred: 421 Service Temporarily Unavailable


Now downgrade libssl:

LNX lindsey:[/tmp] # dpkg -i /tmp/libssl1.1_1.1.0j-1~deb9u1_i386.deb
dpkg: warning: downgrading libssl1.1:i386 from 1.1.1b-2 to 1.1.0j-1~deb9u1
(Reading database ... 70526 files and directories currently installed.)
Preparing to unpack .../libssl1.1_1.1.0j-1~deb9u1_i386.deb ...
Unpacking libssl1.1:i386 (1.1.0j-1~deb9u1) over (1.1.1b-2) ...
Setting up libssl1.1:i386 (1.1.0j-1~deb9u1) ...
Processing triggers for libc-bin (2.28-10) ...


Reset sendmail host-statistics

LNX lindsey:[/tmp] # rm -rf /var/lib/sendmail/host_status/*

Start Sendmail again:

May 21 14:04:54 lindsey sm-mta[9480]: starting daemon (8.15.2): 
SMTP+queueing at 00:10:00
May 21 14:04:55 lindsey sm-mta[9481]: STARTTLS=client, 
relay=cluster8.eu.messagelabs.com., version=TLSv1.2, verify=OK, 
cipher=ECDHE-RSA-AES256-GCM-SHA384, bits=256/256
May 21 14:04:55 lindsey sm-mta[9481]: x4LC4EgN009396: to=<xxxxx.de>, 
delay=00:00:41, xdelay=00:00:01, mailer=esmtp, pri=211212, 
relay=cluster8.eu.messagelabs.com. [46.226.52.98], dsn=2.0.0, stat=Sent 
(ok 1558440296 qp 11470 
server-9.tower-262.messagelabs.com!1558440295!29341!1)


Mail gets out!


I tried about 2 weeks to get this issue fixed with another dh-key (up to 
4096 bits) or ssl-parameters or such things - no success.

Only downgrading libssl solved the problem.


So the recent libssl 1.1.1b-2 has a really major problem.

With best regards
	Jörg Hinz

-- 

Jörg Hinz
Hinz at Linux-Systeme.de
+49 201 - 29 88 311
+49 172 - 7 222 333

Linux-Systeme GmbH
Langenberger Str. 179, 45277 Essen
www.linux-systeme.de
+49 201 - 29 88 30

http://www.xing.com/profile/Joerg_Hinz

Amtsgericht Essen, HRB 14729
Geschäftsführer Jörg Hinz