Bug#944124: apt fails to verify certificate when using https and ocsp stapling

[Andreas Metzler]

On 2019-11-09 David Kalnischkies <david at kalnischkies.de> wrote:
> On Mon, Nov 04, 2019 at 05:49:53PM +0100, Robert Senger wrote:
>> We are running several debian repositories for custom kernel and
>> patched deb packages. We use apache2 on Buster, with https enabled,
>> to serve the repos.
>> 
>> This worked fine, until we decided to enable ocsp stapling in
>> apache2, which runs other vhosts besides the repos.
>> 
>> Since then, apt fails to validate the server's certificate. Error
>> message is:
>> 
>> Fehl:15 https://microscopium.de/repos/apt/debian/common buster/patched Release
>>   Certificate verification failed: The certificate is NOT trusted.
>>   The received OCSP status response is invalid.  Could not
>>   handshake: Error in the certificate verification. [IP:
>>   fd10:2842:f0d1:101:222:4dff:feb8:17c 8000]
>> 
>> Restarting apache2 helps for a while (apt works, at least once), but
>> the error comes up again when apt is run later.

> The (part of the) error message is from libgnutls30:
> | The certificate is NOT trusted. The received OCSP status response is invalid.
> and is the stringyified GNUTLS_E_CERTIFICATE_VERIFICATION_ERROR error returned
> by gnutls_handshake.

Hello,

currently gnutls-cli connects successfully.
I think it is very fishy that restarting the server lets apt succeed for
some time.

> apts https method is a thin gnutls-powered wrapper around our http method, the
> code lives in the TlsFd struct and the UnwrapTLS method, both here:
> https://salsa.debian.org/apt-team/apt/blob/master/methods/connect.cc#L802

> As you can see, we have no code dealing with OCSP directly and we do
> not keep state ourselves by caching responses or some such, so I am
> completely at a lose what apt could be doing wrong here or how its
> behaviour would change over multiple runs.

> So it is either a bug in gnutls or in how we use it; in both cases I hope the
> gnutls maintainers can shine some light on the issue better than I could and
> hence I reassign to them.

GnuTLS is supposed to handle OCSP stapling transparently, without special
client code.
https://gnutls.org/manual/gnutls.html#index-OCSP-stapling
| Since GnuTLS 3.5.1 the client certificate verification will consider
| the [RFC7633] OCSP-Must-staple certificate extension, and will consider
| it while checking for stapled OCSP responses. If the extension is
| present and no OCSP staple is found, the certificate verification will
| fail and the status code GNUTLS_CERT_MISSING_OCSP_STATUS will returned
| from the verification function. 

The apt code seems to be basically ex-client-x509.c
https://gnutls.org/manual/gnutls.html#Client-example-with-X_002e509-certificate-support
however ex-client-x509.c can connect to / verify the example host.

I have not tried apt. But if it really does not succeed finding the
difference to ex-client-x509.c should help.

>> All web tools tell us that certificate installation and ocsp
>> stapling are correct. No other problems with other https clients
>> have been observed so far.

[...]
> It might also help if you could provide a public testcase as
[...]

AFAIUI microscopium.de is a public testcase.

cu Andreas

-- 
`What a good friend you are to him, Dr. Maturin. His other friends are
so grateful to you.'
`I sew his ears on from time to time, sure'