[pkg-gnupg-maint] Bug#790665: Bug#790665: gpg2 fails to import gpg2 created keyring in a gpg1 created keyring

Sun Jul 5 15:10:59 UTC 2015

Hi,

On Wed, Jul 01, 2015 at 01:31:34AM -0400, Daniel Kahn Gillmor wrote:
> A little bit of terminology will be useful for clarification:

Indeed, but frankly that is the very first time I read terms like
"keybox". Okay, now that you mention it I see it in the "keyring/keybox
… created" message from gpg/2, but nowhere else, so I 'naturally'
assumed this is just a raider^Wkeyrings are now called keybox thing…

> > | /tmp/gpg2to1$ gpg2 --no-options --no-default-keyring --keyring gpg1.ring --import gpg2.ring
> > | gpg: WARNING: unsafe permissions on homedir '/tmp/gpg2to1'
> > | gpg: no valid OpenPGP data found.
> > | gpg: Total number processed: 0
> >
> > (exits with code 2 btw)
> 
> I don't expect this to work.  --import works with raw OpenPGP packets on
> stdin, not with keyboxes.  The fact that "simple keyrings" happen to be
> composed of raw OpenPGP packets is a historical artifact that GnuPG
> upstream has warned against taking for granted for several years now.

I don't know where you get the "stdin" from. What I am trying here is
merging a keybox into a simple keyring in a single command, no funky
business with stdin.

As basically every documentation I came across says "gpg --import
[filename]" I kinda expect that to work with whatever file gpg ends up
producing. "OpenPGP packets" on the other hand makes me immediately think
its an internal detail I shouldn't concern myself with and documentation
seems to agree as there is no mention of this either.

> > Importing gpg1.ring into gpg2.ring works.
> > As does --export | --import:
> > | /tmp/gpg2to1$ gpg2 --no-options --no-default-keyring --keyring gpg2.ring --export | gpg2 --no-options --no-default-keyring --keyring gpg1.ring --import
> > | gpg: WARNING: unsafe permissions on homedir '/tmp/gpg2to1'
> > | gpg: WARNING: unsafe permissions on homedir '/tmp/gpg2to1'
> > | gpg: key 76B9B739: "David Kalnischkies <david at kalnischkies.de>" not changed
> > | gpg: Total number processed: 1
> > | gpg:              unchanged: 1
> 
> right.  --export | --import is the expected workflow.
> 
> > This is the same error as printed if I s#gpg2#gpg# btw which is kinda
> > expected, but it would be handy if gpg could deal with gpg2 keyrings
> > – especially as the gpg1.ring into gpg2.ring generates some very
> > interesting output if done by gpg. If that would work (or at least
> > reliably fail with a good message) this might even help your planed
> > transition…
> 
> I think you're saying here that pointing gpg 1.4.x at a keybox with
> --keyring won't work.  This is known, and it's one of the deficiencies
> of the 1.4 branch: 1.4 doesn't understand the keybox format.  this is
> unlikely to change for 1.4.x.  However, if /usr/bin/gpg is eventually
> supplied by 2.1.x (this is the transition you mentioned), then
> /usr/bin/gpg will support --keyring $KEYBOX as well as --keyring
> $SIMPLE_KEYRING.

This side-comment was mainly about partial upgrades and such. For
example, using gnupg (>2.1) and gpgv(<2) together doesn't work that
well. Neither does a *-archive-keyring package which happens to ship
a keybox while you haven't the tools to deal with it. A bunch of breaks
should help with that, but it would be nice if gpg1 could tell something
is wrong rather than giving strange errors nonetheless.

(No idea what your actual transition will look like as all I could find
is Teams/GnuPG/Transition which mentions 3 different possibilities…)

> > Background information: apt tries to sidestep the existing 40 --keyring
> > (and threatened 1 --keyring) limit by first merging all its fragmented
> > key(ring)s shipped by *-archive-keyring packages (and hence might be
> > created by other gpg versions) into one big keyring which is then used
> > as keyring for whatever action is actually supposed to happen.
> 
> hmmm -- if the goal is to merge simple keyrings together into a larger
> simple keyring, then the simplest merge mechanism is /bin/cat:
> 
>   /bin/cat /etc/apt/trusted.gpg.d/*.gpg > $TMPDIR/pubring.gpg

Huh, this (like many of your other suggestions I snipped) is
interesting. I just naturally assumed that simple appending wouldn't
work. Is this a dependable interface or is it just working for now™?

If its the former I would be tempted to declare keyboxes a no-go for
-keyring packages dropping files into that directory and use this + gpgv
to lessen our dependency on gnupg in the future.

After all, that was the plan behind introducing trusted.gpg.d/, to get
to a point where normal operations (-keyring package installation/
removal and Release file verification) do not require gpg anymore.
I dropped that plan after we hit the 40 keyring limit thinking that we
would need gpg to merge them together…

> > This is all fine and dandy for read-only operations as most of apt-key
> > operations are, but you can also add/remove/update keys and that is
> > where it gets complicated as these actions apply on our big merged
> > keyring and have to be split up and applied to the fragmented keys
> > instead.
> 
> Can you give me an example of what these actions are and how they're
> invoked, or point me to the places in the code that you're fighting
> with?  I'm pretty sure there are useful conventions that do what you're
> looking to do, maybe we just need to figure them out.

Well, pre-2.1 state of apt-key is basically:
https://anonscm.debian.org/cgit/apt/apt.git/tree/cmdline/apt-key.in?id=refs/heads/debian/experimental

But lets produce some theory here:

Setup:
* We have a bunch of keyrings: trusted.gpg more or less under our
  control and here for compatibility reasons mostly while distributions
  as well as users are supposed to drop keyrings into trusted.gpg.d/.
  That can be freakishly many, so at some point people were hitting 40
  limit and if 1 becomes a reality… some sort of merging seems needed.
* As far as I understood you so far I can end up with a lovely mixture
  of simple keyrings and keyboxes in this directory depending on what
  gpg version happens to create these keyrings and how. The less
  strict the better usually, but that is a bonus objective.
* apt-key is supposed to handle all the nitty gritty details of working
  with gpg/gpgv/gpg2/gpgv2 in whatever version we have to deal with in
  oldstable and stable (so that (partial) upgrades either way work).

Usecases:
* apt-key verify – undocumented as it is supposed to be used only
  internally by apt to verify Release files (this was hardcoded before
  in libapt directly, I just moved it out for the merging). Wrapping
  around "something" which verifies the given file + detached sig
  (clearsigned files are split up before) and gives some info about
  which is the result of that: good sig, bad sig, unknown key, that sort
  of stuff.

All other usecases are kinda optional, but as this is security related
even the more obscure ones are on a keep-forever list. "update" is on
the way out through (no point of this if a package can just add/remove
entire keyrings on its own). "add" has mostly the same faith, but as it
is you can change keys this way as well (import new expire dates, sigs,
…). Not that you should, but you can. "del" of course should work, even
if rarely used (yes, people complain if they can't remove the debian
archive keyring – or at least some of them… *shrug*).  net-update was
never a thing for Debian and as far as I know it isn't going anywhere
either (better plans mostly).  That leaves the basic readonly wrappers
list, finger(print) and export. And then there is the bad apple 'adv'
which can be used to do all the good/bad things you could come up with.
Mostly used by users to --recv-key from the net into apts trusted
keyring or --refresh-keys. Both are not exactly on my favorite list
either, but people wanna do it and who am I to forbit/break it…

> Is any of this being used for apt's validation of archive signatures, or
> does apt rely on gpgv for signature validation?  all versions of gpgv i
> know about (including 2.1.x) rely on simple keyrings, and cannot use
> keyboxes.  (i just opened https://bugs.gnupg.org/gnupg/issue2025 about
> this).

Well, /sid and earlier all use gpgv exclusive for verification, but that
has the limited number of keyrings problem, so in /experimental "apt-key
verify" is taught to use gpg to first merge the keyrings and then calls
gpgv with this big keyring. Expect that this isn't working well if you
end up on systems with gpg2 used for the merging and gpgv for the
verify, so I am probably going to drop that in favor of just using "gpg
--verify" instead of gpgv all the time. There is only a very minor thing
I tried this gpg/gpgv shenanigan before for which is that apt-cdrom
traditionally shows output from gpgv which doesn't care for trust, while
gpg seems rather hard to be convinced to not care about it (e.g. not
printing "gpg: WARNING: Using untrusted key!" as that is quiet scary),
regardless of how hard you try (--no-auto-check-trustdb --trust-model
always). That was at least the plan until you came along with 'cat'…

> Please give me more details of what you need to do with apt and what is
> failing, and i'll be happy to help you get it sorted out.

So, a bunch of more details as you wished. :)

Just as a summary: apt/experimental works fine with 2.0 already, 2.1 has
these two rather minor issues we can easily solve one way or the other.
It isn't "optimized" yet and your tips give a good idea what to do in
this regard, but at least functional and (hopefully) good enough to make
apt/stretch deal with newer gpg's for the forseeable future (worked
"well" in the 2.0 to 2.1 jump, but I can't be unlucky twice, can I…).

Sidenote: The apt-key above can be told to use gpg or gpg2 and gpg2
(2.1, not 2.0) seems to be considerably slower. Is this known and/or do
you have an idea why this is so? Its not like apt-key operation would be
really time-critical, so usually nobody would notice, but our apt-key
testcase makes it visible as it runs slower…

Best regards

David Kalnischkies
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/pkg-gnupg-maint/attachments/20150705/10844f5b/attachment.sig>