Bug#566351: libgcrypt problem

[Howard Chu]

Simon Josefsson wrote:
> Howard Chu<hyc at highlandsun.com>  writes:
>> App writers using gnutls should not need to know that libgcrypt is
>> under the covers and needs special handling.
>
> The Libgcrypt designers appears to have believed otherwise, and given
> how broken the applications appear to be in this area decision (setuid
> programs doing LDAP+TLS?  Sigh) it seems that was useful because we are
> now aware of these sub-optimal security practices.  Working on fixing
> this to use a better mechanism (userv was mentioned) would be useful
> regardless of what is done (or not) in GnuTLS/Libgcrypt.

Yes, as I pointed out in the Ubuntu bug report, nss-pam-ldapd and/or OpenLDAP 
nssov completely avoids this specific problem, but the problems in 
GnuTLS/libgcrypt remain.

> The crypto parts of OpenSSL have a similar issue: they need mutexes
> provided by the application.  Many libraries using OpenSSL doesn't
> provide an interface to provide these mutexes, but depend on the
> application initializing OpenSSL themselves.

And again in this specific instance, OpenSSL still works correctly with 
nss-ldap, because nss-ldap provides its own mutex locking already. The reason 
people are noticing this problem now is because GnuTLS/libgcrypt *don't* 
behave correctly, even when initialized as documented *and* locking is taken 
care of at a higher level.

>> (Indeed, as illustrated in this bug report, apps generally won't and
>> can't know anything about the underlying libraries.)
>
> That is a problem indeed.
>
>> So aside from deciding what fix if any is appropriate for libgcrypt's
>> secmem implementation, the larger issue remains of how to make
>> libgcrypt safe for use when it's nested under other libraries like
>> gnutls. Saying "applications are responsible for correctly
>> initializing libgcrypt" is a non-starter. libgcrypt needs to have that
>> requirement removed, and gnutls needs to be more comprehensive and
>> explicit in the steps it takes to initialize libgcrypt, so that gnutls
>> callers are completely shielded from the lower API layers.
>
> Right, I think things can be improved here.  However sometimes it is not
> possible to shield applications from lower API layers: compare threads.
> You can't have a multi-threaded application access the entropy functions
> in any reliable manner with Libgcrypt or OpenSSL.  The application needs
> to provide the libraries with mutexes for the thread library it is
> using, so the libraries can protect these accesses properly.

Your example here is a poor one, for a number of reasons. First of all you're 
assuming a single global entropy pool, which is already a design flaw. If you 
associate the entropy pool with a caller context, most of this problem goes 
away. Secondly, on most of the platforms of interest, OpenSSL simply reads 
from /dev/[u]random or EGD/PRNGD, and the read() is an atomic operation and 
doesn't need thread/reentrancy protection.

If you properly partition the library state across context handles and session 
handles most of the need for mutexes disappears. IMO the fact that OpenSSL 
uses context handles and neither GnuTLS nor MozillaNSS do makes OpenSSL 
superior for deployment in real world environments, because many 
libraries/code contexts within one process can use OpenSSL without stepping on 
each other, and that's the default behavior.

Anyway, probably the first step for GnuTLS is to add a new init API that also 
initializes gcrypt threading, so that apps do all of their interaction solely 
with GnuTLS, instead of having to also use gcrypt init APIs and hope they 
actually coordinate correctly.
-- 
   -- Howard Chu
   CTO, Symas Corp.           http://www.symas.com
   Director, Highland Sun     http://highlandsun.com/hyc/
   Chief Architect, OpenLDAP  http://www.openldap.org/project/