[pkg-gnupg-maint] Upstream request: Please use the default keyservers

[Daniel Kahn Gillmor]

Hi Andre--

Thanks for writing!  I appreciate your engaging on this, and i
appreciate all your work for the ecosystem around these issues and
concerns.

I'm cc'ing Kristian Fiskerstrand here, as he is another respected member
of the community who has contributed years of labor, machine resources,
thought and care into the SKS pool.  I think he is probably interested
in these issues too, and would have a good perspective to share on the
topic.

On Wed 2020-02-26 13:22:28 +0100, Andre Heinecke wrote:

> yesterday I installed debian buster and was unable to find my own key.
>
> Turns out that debian is patching in a centralized keyserver instead of using 
> the decentralized standard sks keyserver network.
>
> https://salsa.debian.org/debian/gnupg2/-/blob/debian/buster/debian/patches/keyserver-cleanup/Use-hkps-keys.openpgp.org-as-the-default-keyserver.patch
>
> Please remove that patch.

That patch is in place because the SKS keyserver network preferred by
upstream by default will inflict major costs on users, including when
they fetch or update certain certificates.

I think you already know about this, but for the record in this
conversation, the SKS keyserver network as it currently exists is
vulnerable to a host of attacks, some of which are detailed here:

   https://tools.ietf.org/html/draft-dkg-openpgp-abuse-resistant-keystore-04

The SKS pool has not taken any of the recommended mitigations in that
draft thus far, as far as i know.

So, a user who fetches a flooded certificate from the SKS keyserver
network might incur dozens of megabytes of traffic (and a
not-insignificant amount of CPU time) in exchange for very little
(possibly no) additional data.

Users who are refreshing their full keyring regularly (as recommended,
to check for revocations or subkey rotation) would see their risks
increase proportional to the size of their local keyring.

As /usr/share/doc/{gnupg,gpg,gpgconf}/NEWS.Debian.gz says:

-------------------
  Upstream GnuPG now defaults to not accepting third-party certifications
  from the keyserver network.  Given that the SKS keyserver network is
  under attack via certificate flooding, and third-party certifications
  will not be accepted anyway, we now ship with the more tightly-constrained
  and abuse-resistant system hkps://keys.openpgp.org as the default
  keyserver.

  Users with bandwidth to spare who want to try their luck with the SKS
  pool should add the following line to ~/.gnupg/dirmngr.conf to revert to
  upstream's default keyserver:

      keyserver hkps://hkps.pool.sks-keyservers.net

  See the 2.2.17 section in the upstream NEWS file at
  /usr/share/doc/gnupg/NEWS.gz for more information about fully
  reverting to the old, risky behavior.
-------------------

> This is an opinion based topic. Debian should not patch software because it 
> has a different opinion then upstream.

Sorry, but i don't see the distinction between "opinion-based" and
"fact-based" here.  As one of the maintainers of the GnuPG package in
debian, I need to think about the impact that these choices have on
users of GnuPG in Debian, and about the impact of Debian users on the
rest of the OpenPGP ecosystem.

Interacting with the SKS pool implies a significant increase in both
bandwidth and CPU use.  Furthermore, deployment considerations and
experimental testing shows that queries to the SKS pool take longer to
complete on average (both with and without the use of Tor, even for
non-flooded, minimal cetificates), and are more likely to return
unreliable answers (e.g. gateway failures between nginx and the backing
sks server, due to sks's non-concurrent nature and choice of bdb as
backing store).

This unreliability for SKS is both confounded by and compounded by
dirmngr's overaggressive negative caching policy for server availability
(see https://dev.gnupg.org/T4513) -- users are more likely to get
stochastically bad results when querying the SKS pool from dirmngr.

It is certainly my opinion that this set of facts represents a bad
tradeoff for users and the ecosystem as a whole.  But i consider my role
as a distribution package maintainer to take user and ecosystem concerns
seriously even when upstream can't prioritize those concerns.

Those differences of opinion can be quite wide-ranging.  For example,
we've had user concerns about battery drain from unnecessary daemon
wakeups, and we have patched those out in debian while it was feasible
to do so, despite upstream being uninterested in that problem.  (i
believe gniibe did upstream some of the work recently, thank you!)
We've also adopted some trivial changes that make it safer and easier to
modify the source that upstream sees no need for
(https://dev.gnupg.org/T4593).  And for nearly a year now, we've handled
importing updates of revocation information and subkey rotation
information for certificates even if the incoming certificate-like-thing
has no User IDs (https://dev.gnupg.org/T4593), a problem which upstream
seems unconcerned about.  Similarly, we've limited ptrace access to RAM
of gpg-agent and scdaemon, even though upstream doesn't consider this
mitigation as significant or worth making
(https://bugs.debian.org/712744).  And debian adopted changes to the
cryptographic defaults well in advance of upstream, as an attempt to
mitigate the increasing power of both cryptanalysis and large-scale
computing farms.

So yes, there are differences of opinion, but i don't think it's correct
to say that Debian should not patch software because it has a different
opinion than upstream.

> There is a lot of rationale against the SKS Network, but there is also a lot 
> of rationale against a centralized keyserver, which introduces a single point 
> of attack, leaks information about key queries to a single instance etc. etc. 
> Esp. since debian is sensitive about privacy and we have for example disabled 
> auto-key-retrieve by default on your request (where we also agreed). This 
> patch is completely the opposite of that.

As you know, i have also helped to operate one of the keyserver that
used to be in the SKS pool for years, and i share your affinity for
decentralized systems.

I agree with you that centralized keyservers are super problematic from
a privacy perspective, and with your concern about a single-point of
attack.  I'm not sure what to think about "etc etc" ;) but i'd assume
that your additional concerns include robustness in the face of a legal
or technical adversary as well.

However, we should look at the SKS pool with clear eyes on the same
terms.

In terms of single-point-of-attack, Kristian's role is hard to ignore.
If Kristian were compromised or incapacitated, his DNS service suborned,
or his scripts that measure inclusion in the pool taken over, the SKS
pool would certainly go down or could be redirected.

In terms of information leakge to a single instance, the default SKS
pool (hkps.pool.sks-keyservers.net) is not particularly decentralized:
according to https://sks-keyservers.net/status/ today, only three nodes
answer to hkps:

 - sks.pod02.fleetstreetops.com
 - pgpkeys.eu
 - pgpkeys.co.uk

the latter two hosts are controlled by the same entity (ironically, the
link listed for the controlling entity on the status page,
https://sks-keyservers.net/pks/lookup?op=get&search=0xEBF54CD6472EFFFF,
returns a 502 Bad Gateway error at the moment), and the first host only
has an IPv4 address.  So an IPv6-only user would not spread their
searches across multiple unaffiliated parties at all (both IPv6
addresses go to the same party), and dual-stack hosts that selects
randomly among the 5 available IP addresses will still send 80% of all
their traffic to the same party.

Addressing the technical robustness concerns -- a distributed network
*can* offer more technical robustness if it is implemented with an eye
toward fault tolerance.  Unfortunately, the SKS network hasn't made
strides to do that; performance problems have been worked around for
years (i know -- i've helped work around them!) but the core issues
(single-threaded, synchronous sks process; the archaic database backend)
haven't been addressed.  keys.openpgp.org has seen significant attention
to engineering efficiency and reliability, which results in the
experimental results described above.

As for concerns about robustness against legal attack: the SKS network
has seen participants drop off out of fear (warranted or not) about GDPR
and similar privacy concerns -- there is a not insubstantial perception
of legal risk associated with running a global, append-only datastore.
The SKS pool also has no stated privacy policy, while keys.openpgp.org,
despite its flaws, has at least formulated
https://keys.openpgp.org/about/privacy, and has made several clear
public decisions based on those concerns, which lead me to think it will
have a clearer and simpler response to any legal attack based on claims
of privacy violations.

> There is not even an instiution like the GnuPG e.V. behind this service, it 
> might change at a whim.

There is no institution like GnuPG e.V. behind the SKS keyserver pool
either, as far as i'm aware.  Am i wrong about that?  If there is an
institutional backing for the SKS keyserver pool, does that mean that
the institutional backer is responsible for any legal attack against the
pool generally?

> As distributor of Gpg4win I am also facing keyserver issues, but for now we 
> don't have better alternatives. That is why GnuPG still has it as default. We 
> want a decentralized hokeypuck network but keys.openpgp.org is definetly a step 
> in the wrong direction. Please trust the GnuPG project on that, even though 
> your personal opinion might differ.

I'm looking for data which would change my opinion.  I would be very
happy to change my mind, and i would readily adopt an actually
decentralized system that offers the types of robustness and user-driven
features that are described above.  But i haven't seen such a system.

Furthermore, other distributions of GnuPG (e.g. gpg-tools on Mac OS X)
have also switched to keys.openpgp.org.

> Patching the man page makes it appear for debian users that the GnuPG Project 
> is supporting keys.openpgp.org or thinking that using a central server is a 
> good idea. We do not.

That's a good point, thanks for raising this.  I've updated our patch so
that the dirmngr manpage makes it clear that this is a debian-specific
change.  You should see that in the dirmngr(1) manpage that ships in
dirmngr 2.2.19-2.

I'm as sad as you are about the fate of SKS, but we need to acknowledge
that it was designed for a different network, without any appreciable
defense in depth, and hasn't matured to the level where it is a
responsible default.

GnuPG users on Debian need to be able to robustly and reliably fetch
peer certificates, updates, and revocations, and they need the service
that they use to not be susceptible to trivial flooding attacks by third
parties that make any certificate (or even worse, any revocation
notification) unavailable.  The SKS pool is no longer fit for those
purposes (and perhaps it never was, but now it is visibly failing), so i
don't think we can ship GnuPG in debian with that default in place in
good conscience.

Regards,

        --dkg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-gnupg-maint/attachments/20200227/ecb89172/attachment.sig>