[Pkg-openssl-devel] Bug#829108: libssl-dev: HMAC_Init_ex incompatible change (possibly doc bug)

[Marc Lehmann]

Package: libssl-dev
Version: 1.0.1t-1+deb8u2
Severity: normal

Dear Maintainer,

the HMAC manpage states:

   HMAC_Init_ex() initializes or reuses a HMAC_CTX structure to use the
   function evp_md and key key. Either can be NULL, in which case the
   existing one will be reused.

However, the current code does not allow the key to be zero when evp_md is
non-zero in all cases:

    /* If we are changing MD then we must have a key */
    if (md != NULL && md != ctx->md && (key == NULL || len < 0))
        return 0;

That means contrary to the documentation, the existing salt isn't reused
when the md argument is non-zero (and changes).

The issue is corrobated by the fact that HMAC_init_ex only relatively
recently gained a status return, so older programs won't check the return
value and will continue on, getting wrong results.

This particular line was introduced with this change:

https://github.com/openssl/openssl/commit/4b464e7b46682f568a5df550426b0cf4b22e2485

Although I don't know whether this just reworks the logic or introduces
the problem in the first place.

One program that might to be affected is GVPE - I opened a bug report
about it no longer working, although I can't find it at the moment. It
is possible (but not certain) that this is the reason for it no longer
working. Even though GVPE has had return code checking, due to a glitch
it wasn't enabled before openssl 1.1.0, so would not trigger with 1.0.x
builds.

So, either:

a) this is an incompatible and unintended change. in this case, there is potential
   for programs silently failing to compute correct hmacs now.
b) this is an incompatible but intended change, in which case this is a documentation
   bug.
c) this is not an incompatible recent change, in which case the logic always
   was like this but was reworked. in this case it is a documentation bug as well.
d) it is intended behaviour and the previous behaviour
   wasn't correct (e.g. it didn't reuse the key, but did something else).
   also a documentation bug in this case.

If this is an unintended behaviour change, maybe the large scale API
breakage in 1.1.0 would be a good moment to also rename HMAC_init_ex, so
programs have a chance to adaot.

Greetings,

-- System Information:
Debian Release: 8.5
  APT prefers stable
  APT policy: (990, 'stable'), (500, 'unstable'), (500, 'testing'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.4.14-040414-generic (SMP w/8 CPU cores)
Locale: LANG=C, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Init: systemd (via /run/systemd/system)

Versions of packages libssl-dev depends on:
ii  libssl1.0.0  1.0.1t-1+deb8u2
ii  zlib1g-dev   1:1.2.8.dfsg-2+b1

Versions of packages libssl-dev recommends:
ii  libssl-doc  1.0.1t-1+deb8u2

libssl-dev suggests no packages.

-- no debconf information