[pkg-gnupg-maint] Bug#927336: Bug#927336: after buster upgrade (2.1.18-8~deb9u3 -> 2.2.12-1) --search-keys stops working due to dirmngr/keyserver/tor problem: add NEWS?

Fri Apr 19 16:32:16 BST 2019

Control: tags 927336 + moreinfo upstream

Hi Tomas--

Thanks for all the details in this report, and i'm sorry that you ran
into trouble with your upgrade.

You've identified a few different interlocking issues here, and I've
tried to parse them out separately, and i've documented some of them as
work for upstream, as i've noted below.  I'm not sure what to focus on
concretely for this report, so i'm asking you for more feedback to help
scope it and make it something solvable.

On Thu 2019-04-18 09:09:20 +0200, Tomas Pospisek wrote:
> TLDR; please tell the user how to migrate from jessie to buster.

i was hoping this would be:

    apt update && apt full-upgrade

and then reboot into the new kernel :)

>     $ gpg --search-keys 1397BC53640DB551
>     gpg: WARNING: Tor is not running
>     gpg: error searching keyserver: Connection refused
>     gpg: keyserver search failed: Connection refused

Alas, the keyserver ntework generally is failing these days (see for
example discussion over here:
https://mailarchive.ietf.org/arch/msg/openpgp/fH29WI7QLmaN3gO-T222G8LPCNE)

> Based on the above warning I guessed the problem would be that `tor` is
> not running. Since there's already *way* too much bloat in the form of
> unasked for daemons running on my Debian system, I have tor disabled.

If there is no listening tor daemon, then dirmngr should not be trying
to use tor at all.  It is expected to be autodetected at dirmngr
startup.  If that autodetection is not happening correctly, then maybe
we should retitle this bug report and try to tackle it specifically.

If what happened was:

 * dirmngr started in autodetection mode while tor was running
 * you terminated and disabled the tor daemon
 * dirmngr now refuses to connect

then you can solve this problem just by stopping dirmngr. When it
restarts (at the next time gpg needs to access the network, it will be
launched automatically) it will autodetect the lack of tor and work as
expected.

For people who don't manually disable their tor daemon as it sounds like
you did, they don't even need to restart dirmngr.

I've started a discussion about improving the tor autodetection behavior
with upstream:

   https://dev.gnupg.org/T4465

Feel free to chime in there if you want to help make this smoother in
the long-term.

>     keyserver keyserver.ubuntu.com
>
> from `~/.gnupg/gpg.conf` and insert it into `.gnupg/dirmngr.conf`. That
> still did not do the trick. The URL had also to be adapted. So in the
> end:
>
>     $ cat ~/.gnupg/dirmngr.conf
>     no-use-tor
>     keyserver hkp://keyserver.ubuntu.com

The recommended configuration for gpg and company is no configuration at
all, where possible.  dirmngr should have a sensible default keyserver
option built-in, and you shouldn't need to specify keyserver anywhere.
This wasn't always the case, sadly, but it has been since debian stretch
was released.

If you do specify the keyserver explicitly, though, then i agree that
you're responsible for maintaining it.  the keyserver option in gpg.conf
was deprecated before stretch was released (it belongs in dirmngr.conf
for modern GnuPG), so i'm not sure why this is an issue specifically for
stretch to buster.

I've opened this report upstream to make it easier for folks who have
legacy bare hostnames lying around in their config:

    https://dev.gnupg.org/T4467

And i've opened this report about problematic documenation for gpg's
legacy `--keyserver` option:

    https://dev.gnupg.org/T4466

If you know anyone who wants to propose patches for either of those,
please let me know, i'd be happy to shepherd them through upstream, or
ship them in debian at least if upstream is slow picking them up.

> It also seems relevant to note, that dirmngr is yet another daemon
> constantly running, binding system resources for very low user benefit
> (I must be using `gpg --search-keys` or `gpg --recv-keys` about twice
> a year).

If you don't actually ever use dirmngr, then it will never be launched.
If that's not the case for you, please describe your system
configuration so we can debug it.

That said, you should be using dirmngr to refresh your keyring more
regularly, otherwise you'll never know about revocations or expirations.

Once dirmngr is running but idle, i'd like to identify what specific
resources it's consuming.  We should identify any significant resources
and tune dirmngr so that in its idle state it consumes less.

But i don't think having a running, idle process (it seems to sit in a
lengthy pselect6() loop on my system) is on its own a lot of resources
to be worried about.  Its job is to maintain state about knowledge of
the network, and it's not unreasonable to do that in the memory of a
mostly-idle backgrounded process.  What resource consumption are you
worried about specifically?

> Since dirmngr won't detect a config file change,

autodetecting a config file change is problematic in many situations,
like half-edited files that are saved mid-thought but not in a coherent
state, so i don't think that is a useful approach, sadly.

If we could enforce configuration file changes to happen
programmatically (e.g. via gpgconf) then i could see it, but lots of
people edit their gpg configs by hand.

> it needs to be killed. It's reaction to a SIGTERM or SIGHUP seems to
> be erratic.  It just as often terminates as it does not. So it needs
> to be SIGKILLed to pick up the new config, which makes the whole
> proces even more burdensome.

this is a separate discussion, and i'd be happy to have it elsewhere,
especially with more details.  The dirmngr documentation hints (buggily)
that SIGHUP is insufficient to get dirmngr out of the `use-tor` state.
I'm trying to get that fixed too:

   https://lists.gnupg.org/pipermail/gnupg-devel/2019-April/034301.html

> Now finding all these took me about half an hour (and I'm not sure
> whether they're correct and/or relevant). Please decide on a mechanism
> to let the user know how to migrate his GPG setup into a working
> condition again. I think the NEWS file would be the ideal place, however
> maybe a note in the Release Notes would also be possible.

I welcome suggestions for specific text for either
/usr/share/doc/gnupg/NEWS.gz or /usr/share/doc/dirmngr/NEWS.gz, but i
don't think the debian packaging should encourage people to either:

 * explicitly disable the use of tor, which i think is a useful
   privacy-preserving measure.

 * use a non-default keyserver unless they know what they're doing.

What do you think we should do?

all the best,

     --dkg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-gnupg-maint/attachments/20190419/eb3ff408/attachment.sig>