[Pkg-privacy-maintainers] Bug#836266: Bug#836266: Bug#836266: Bug#836266: Bug#836266: dirmngr: Please disable "use-tor" by default.

[Antoine Beaupré]

On 2017-01-10 18:28:59, Daniel Kahn Gillmor wrote:
> On Tue 2017-01-10 14:15:43 -0500, Antoine Beaupre wrote:
>> As things stand now, I see no choice but to stop using parcimonie, which
>> means:
>>
>>  1. i will not update my keyring in a timely manner anymore, or;
>>  2. i will reveal my keyring social graph to the keyserver and
>>     possible attackers
>>
>> That seems like the opposite of what parcimonie is trying to
>> accomplish.
>
> fwiw, the ideal long-term fix here is that the logic of parcimonie gets
> folded into dirmngr itself, and just works automatically if tor is
> available.
>
> i've got sketches for how this would work if anyone has the time to work
> on it.
>
> Please see https://bugs.gnupg.org/gnupg/issue1827 for more details.

We're now a year and a half later and I've upgraded my box to Debian
Buster. Remembering this issue (and that I had no mechanism for key
refresh still!) I figured I would give it a shot again. So I installed
parcimonie and let it run for a while. It seems to do its thing but it's
hard to tell as I'm not sure if there are logs anywhere.

That said, `use-tor` is still as problematic as it was back in
stretch. As a random example, I tried to search for a key on a
keyserver, and gpg just failed to get anything. The diagnostics are, as
usual, fairly limited, but the error Phil Morrell got is pretty typical:

       Failed to fetch key 6ACBAD6A729326258CF725C6E7519C8D747F00DC: gpg: keyserver receive failed: No data

Since it's not the first time I have had this problem, I have full debug
enabled in dirmngr (hello metadata leakage!). This translate into this
error:

2018-08-27 21:20:17 dirmngr[9652.6] erreur d'accès à « https://193.164.133.100:443/pks/lookup?op=index&options=mr&search=milan%40debian%2Eorg » : état HTTP 502
2018-08-27 21:20:17 dirmngr[9652.6] command 'KS_SEARCH' failed: Pas de données
2018-08-27 21:20:17 dirmngr[9652.6] DBG: chan_6 -> ERR 167772218 Pas de données <Dirmngr>

What's dumb here is that dirmngr doesn't fallback to other
servers. Repeated attempts to search keys will fail with the same
incomprehensible and useless error message. ("HTTP status 502" would
actually be more useful for debugging, for example, along with which
host is responsible - but GnuPG only blesses us only with "no data".) So
the bug in dirmngr here at least is that it doesn't rotate to the next
available server in the pool when failing with tor.

The workaround, as Morrell explained, is to restart dirmngr which
magically picks another host and moves on.

I know this is not Parcimonie's fault. It's gnupg's fault or, more
precisely, dirmngr's, but it seems difficult to change things over
there: this would require rewriting dirmngr's network routines or
reimplementing parcimonie within dirmngr itself.

After spending years fighting with GnuPG at various levels, neither
looks very attractive or accessible to me as a developer right now.

Instead, I've started thinking about what a parcimonie rewrite would
look like, one that would *not* depend on dirmngr (or, in fact, any
specific OpenPGP implementation). If you permit, I would like to use
this space to brainstorm such a design, which can be broken up into the
following step:

 1. build a random list of keys to fetch, idempotently
 2. talk with Tor
 3. fetch keys from a keyserver
 4. validate the keys (!)
 5. reinject in the data store

In details, it would look something like this:

 1. The first step is to enumerate keys. This requires talking to the
    keystore: it can be done with gpg --export but that means a lot of
    data. Parsing the `--list-keys --with-colons` output might be
    mandated here, as much as it hurts me to even think about this. This
    would load a list of fingerprints to refresh. This list should be
    sorted internally, and then copied and shuffled so we have a list of
    keys to iterate over reliably. This process can be repeated after a
    timeout: new keys would be added, sorted, to the sorted list and
    then added in a random location in the shuffled list.

 2. Fetching Tor is not that complicated, and is the cornerstone of this
    program. Talking to it is simply like talking with a SOCKS5 proxy,
    something which Python requests supports since 2.10, if my memory
    serves me right (jessie-backports and above).

 3. Then parcimonie also needs to talk to keyservers: right now it lets
    gnupg do the talking, but this is actually fairly easy as well, as
    HKP was implemented on top of a few HTTP verbs. I have implemented
    such a client in the PGPy library in a few lines of code:

    https://github.com/SecurityInnovation/PGPy/blob/d5e46733df34f14f83bda5ed2bc0bcc13bd971e3/pgpy/types.py#L267

 4. This actually parses the packet as well and this is where things get
    a little more complicated: what's an acceptable response from a
    keyserver?  This is another thing that's delegated to GnuPG right
    now, but it would be interesting to formalize this and (self-?)
    authenticate the key material. Or can we delegate *just* that bit to
    GnuPG?

 5. Then the last step is simply to feed the resulting key material back
    into the keystore, either gpg --import or whatever backend we want
    supported.

All this doesn't seem that complicated to me. The tricky bit is the gate
to keep garbage and hostile keys from going into the keyring. It would
probably be where the rubber meets the road, as there's an incredible
variety of stuff on keyservers. Not that many programs or libraries can
parse this reliably or at all, GnuPG included. PGPy, in particular, is
not very well tooled for this just yet and cannot correctly parse ECC
keys, for example.

I would welcome feedback on how this could be done, or if it's just an
incredibly stupid idea.

Thanks, and sorry for hijacking this thread with such wild ideas. :)

A.

-- 
Rock journalism is people who can't write, interviewing people who can't
talk, in order to provide articles for people who can't read.
                        - Frank Zappa