[pkg-gnupg-maint] Bug#790665: Bug#790665: gpg2 fails to import gpg2 created keyring in a gpg1 created keyring

[Daniel Kahn Gillmor]

Control: affects 790665 apt
Control: retitle 790665 improve gpg 2.1.x support in apt (gpg2 --import does not work on keyboxes)
Control: severity 790665 wishlist
Control: tags 790665 + moreinfo

Hi David--

On Tue 2015-06-30 13:22:59 -0400, David Kalnischkies <david at kalnischkies.de> wrote:

> while continuing my work on gpg2-proofing apt-key (and hence apt)
> I noticed that the 2.1 branch (2.0 isn't effected) currently in
> experimental can't import (even if it has nothing to do) from a gpg2
> created keyring, if the target is a gpg1 created one.

Thanks for reporting this, and thanks even more for your work on making
sure apt is good to go with gpg 2.1.x.  I want to help with that any way
i can.

A little bit of terminology will be useful for clarification:

What you've called "gpg1 created keyring" is i think a simple linear
series of OpenPGP packets.  It'll be easier if we call this a "simple
keyring".

What you've called a "gpg2 created keyring" is actually a keybox, which
is a more complicated, indexed data structure that has much faster read
access time. It'll be easier if we call this a "keybox".

The first half of this ticket basically says "'gpg --import' doesn't
work when stdin is a keybox", which is true for all supported branches
of gpg -- 1.4.x, 2.0.x, and 2.1.x.  Since --import was never intended to
work with keyboxes on stdin, this is neither surprising nor a problem.

Let's dig a little deeper:

> Consider the following example:
> | ~$ export LANG=C.UTF-8
> | ~$ export GNUPGHOME="/tmp/gpg2to1"
> | ~$ mkdir $GNUPGHOME; cd $GNUPGHOME

In the examples below, you use short Key IDs to identify a key -- i'm
sure you know this is a bad idea (if not, please read
https://www.debian-administration.org/users/dkg/weblog/105).  Assuming
that this is for convenience; it's just as convenient and
cryptographically stronger to declare a stronger full
fingerprint here and use it later as $KEY:

 export KEY=0x1E09B66DED7B5875B552B142E197012676B9B739

> | /tmp/gpg2to1$ gpg --no-options --no-default-keyring --keyring ~/.gnupg/pubring.gpg --export 76B9B739 | gpg --no-options --no-default-keyring --keyring ./gpg1.ring --import
> | gpg: WARNING: unsafe permissions on homedir `/tmp/gpg2to1'
> | gpg: WARNING: unsafe permissions on homedir `/tmp/gpg2to1'
> | gpg: keyring `/tmp/gpg2to1/secring.gpg' created
> | gpg: keyring `./gpg1.ring' created
> | gpg: /tmp/gpg2to1/trustdb.gpg: trustdb created
> | gpg: key 76B9B739: public key "David Kalnischkies <david at kalnischkies.de>" imported
> | gpg: Total number processed: 1
> | gpg:               imported: 1  (RSA: 1)
> | gpg: no ultimately trusted keys found

I'd call this "creating a simple keyring", but it's much easier to just
do the following:

  gpg --no-options --no-default-keyring --keyring ~/.gnupg/pubring.gpg --export "$KEY" > ./gpg1.ring

> | /tmp/gpg2to1$ gpg2 --no-options --no-default-keyring --keyring ~/.gnupg/pubring.gpg --export 76B9B739 | gpg2 --no-options --no-default-keyring --keyring ./gpg2.ring --import
> | gpg: WARNING: unsafe permissions on homedir '/tmp/gpg2to1'
> | gpg: WARNING: unsafe permissions on homedir '/tmp/gpg2to1'
> | gpg: keybox './gpg2.ring' created
> | gpg: starting migration from earlier GnuPG versions
> | gpg: porting secret keys from '/tmp/gpg2to1/secring.gpg' to gpg-agent
> | gpg: migration succeeded
> | gpg: key 76B9B739: public key "David Kalnischkies <david at kalnischkies.de>" imported
> | gpg: Total number processed: 1
> | gpg:               imported: 1
> | gpg: no ultimately trusted keys found

I'd call this "creating a keybox", but it's probably even easier and
quicker to do the following if you just want a brand new keybox (not
trying to merge into an existing keybox) and don't care about the rest
of the gpg trappings:

  gpg2 --no-options --no-default-keyring --keyring ~/.gnupg/pubring.gpg --export "$KEY" | kbxutil --import-openpgp > gpg2.ring

If you want to convert a keybox into a simple keyring, you can do:

  gpg2 --no-options --no-default-keyring --keyring $KEYBOX --export > $SIMPLE_KEYRING

> | /tmp/gpg2to1$ gpg2 --no-options --no-default-keyring --keyring gpg1.ring --import gpg2.ring
> | gpg: WARNING: unsafe permissions on homedir '/tmp/gpg2to1'
> | gpg: no valid OpenPGP data found.
> | gpg: Total number processed: 0
>
> (exits with code 2 btw)

I don't expect this to work.  --import works with raw OpenPGP packets on
stdin, not with keyboxes.  The fact that "simple keyrings" happen to be
composed of raw OpenPGP packets is a historical artifact that GnuPG
upstream has warned against taking for granted for several years now.

> Importing gpg1.ring into gpg2.ring works.
> As does --export | --import:
> | /tmp/gpg2to1$ gpg2 --no-options --no-default-keyring --keyring gpg2.ring --export | gpg2 --no-options --no-default-keyring --keyring gpg1.ring --import
> | gpg: WARNING: unsafe permissions on homedir '/tmp/gpg2to1'
> | gpg: WARNING: unsafe permissions on homedir '/tmp/gpg2to1'
> | gpg: key 76B9B739: "David Kalnischkies <david at kalnischkies.de>" not changed
> | gpg: Total number processed: 1
> | gpg:              unchanged: 1

right.  --export | --import is the expected workflow.

> This is the same error as printed if I s#gpg2#gpg# btw which is kinda
> expected, but it would be handy if gpg could deal with gpg2 keyrings
> – especially as the gpg1.ring into gpg2.ring generates some very
> interesting output if done by gpg. If that would work (or at least
> reliably fail with a good message) this might even help your planed
> transition…

I think you're saying here that pointing gpg 1.4.x at a keybox with
--keyring won't work.  This is known, and it's one of the deficiencies
of the 1.4 branch: 1.4 doesn't understand the keybox format.  this is
unlikely to change for 1.4.x.  However, if /usr/bin/gpg is eventually
supplied by 2.1.x (this is the transition you mentioned), then
/usr/bin/gpg will support --keyring $KEYBOX as well as --keyring
$SIMPLE_KEYRING.

> Background information: apt tries to sidestep the existing 40 --keyring
> (and threatened 1 --keyring) limit by first merging all its fragmented
> key(ring)s shipped by *-archive-keyring packages (and hence might be
> created by other gpg versions) into one big keyring which is then used
> as keyring for whatever action is actually supposed to happen.

hmmm -- if the goal is to merge simple keyrings together into a larger
simple keyring, then the simplest merge mechanism is /bin/cat:

  /bin/cat /etc/apt/trusted.gpg.d/*.gpg > $TMPDIR/pubring.gpg

This also works for merging these files into whatever gpg's preferred
keyring format (simple keyring for 1.4.x and 2.0.x, keybox for 2.1.x),
if you want to do that:

  /bin/cat /etc/apt/trusted.gpg.d/*.gpg | gpg --no-options --no-default-keyring --keyring $TMPDIR/pubring.ring

(regardless of whether gpg is supplied by 2.1.x or 1.4.x)

> This is all fine and dandy for read-only operations as most of apt-key
> operations are, but you can also add/remove/update keys and that is
> where it gets complicated as these actions apply on our big merged
> keyring and have to be split up and applied to the fragmented keys
> instead.

Can you give me an example of what these actions are and how they're
invoked, or point me to the places in the code that you're fighting
with?  I'm pretty sure there are useful conventions that do what you're
looking to do, maybe we just need to figure them out.

> Add and remove are simple list comparisons, but updated keys (new
> signatures or expire dates) are dealt with by "--import-options
> merge-only --import big.ring" and that is what fails here – even
> through it will be a glorified no-op 99,9% of the time in apt-key.

Are you talking about "apt-key update" here or something else?

Is any of this being used for apt's validation of archive signatures, or
does apt rely on gpgv for signature validation?  all versions of gpgv i
know about (including 2.1.x) rely on simple keyrings, and cannot use
keyboxes.  (i just opened https://bugs.gnupg.org/gnupg/issue2025 about
this).

Please give me more details of what you need to do with apt and what is
failing, and i'll be happy to help you get it sorted out.

Happy hacking,

     --dkg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 948 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-gnupg-maint/attachments/20150701/ef34562d/attachment.sig>