[pkg-go] Security support for packages written in Go

[Michael Hudson-Doyle]

Bit of a late reply, been busy.

On 5 April 2016 at 19:27, Florian Weimer <fw at deneb.enyo.de> wrote:
> Hi,
>
> we need to discuss how we can support applications written in Go for
> stretch.
>
> The most radical approach would be not to ship any Go applications in
> stretch, only the basic Go language implementations.  This is probably
> not desirable.

Yeah, probably not desirable. I think it makes sense to be fairly
selective about which Go applications are supported, though.

> So we need something to deal with the static linking issue.

There is another approach to the static linking issue, which is to
start using dynamic linking instead. It's implemented upstream for
most architectures now (only mips64 le/be and ppc64 be missing I
think). I'm going to be working on starting to use dynamic linking
during the next cycle of Ubuntu development, and I'd certainly be
interested in getting it going for Debian too. (the timeframes re:
stretch release look reasonable for this).

> The current state seems to be that -dev packages do not contain object
> code.  This means that they can be built in any admissible order (as
> determined by their (versioned) build dependencies), without altering
> the end result, which simplifies matters somewhat (but see below).
>
> One approach would be to ship applications as source code only (just
> like libraries), and compile them locally upon installation.  This is
> what Emacs and Common Lisp implementations already do.  With the
> growth of Go compilation and link times, this seems less and less
> attractive, though.

Uh yeah, let's not do this (it also means that any system that wants
to use a Go binary needs a Go development environment installed).

> The alternative is to rebuild reverse dependencies as needed.  I can
> see two challenges with that.  Right now, the Built-Using field only
> records the source versions of the *direct* dependencies (based on the
> dh_golang manual page and a few examples I looked at).

Yeah, that seems like a bug, basically, given my understanding of what
Built-Using is supposed to mean.

Summarising other posts in the thread, it seems the sanest fix for
this goes along the lines of:

1) Change all binary golang-*-dev packages to have a Go-Import-Path
control field (dh_golang can probably add this?)
2) Change dh_golang to use "go list -f {{ .Deps }}" and "dpkg-query -W
-f '${Go-Import-Path}' golang-*-dev" and some perl to generate a
better Built-Using.

We could do it without 1) and the consequent re-uploading of every go
library by using dpkg-query --search a lot, which would be slow I
guess, but maybe could be done as a fallback?

Someone (tm) should file a bug about this, if there is not one already.

> If a critical
> update happens farther down the dependency chain, a tool based on
> Built-Using will not mark the top-level package as a rebuild
> candidate.  When performing the rebuild, it is possible to compensate
> for that by rebuilding everything that has an outdated Go source
> package in its Build-Using field, iteratively, until we reach a
> fixpoint.  But this does not currently work because the -dev packages
> do not contain Built-Using information.

And -dev packages should not really contain Built-Using information...

> But this requires that the source package version changes as a result
> of the rebuild.  I'm not sure if this is what happens.  Ideally, we
> want to do this with binNMUs, not sourceful uploads.

I'm mostly an Ubuntu developer so obviously all uploads change the
source package version :-)

> We may also need
> some build ordering optimization to avoid redundant rebuilds.  This
> means that the fact that the -dev packages contain no object files is
> of little practical consequence: The optimization would have to order
> the builds in the same way as if they did contain object code.  But
> maybe the optimization is not needed.

It would be interesting to check out some imaginary situations (e.g. a
security fix in golang-golang-x-crypto) and see how much this matters
in practice. My suspicion is not all that much, but who knows.

> It does not seem possible to determine rebuild candidates based on
> Built-Using alone, building the transitive closure after the fact.  It
> may have changed between the original application build and subsequent
> library builds.

Yes. So let's fix the problem in the right place :-)

> Unrelated to all that, we cannot currently perform binNMUs in the
> security archive because it does not contain a completely copy of the
> main archive.  I'm not sure if there are good approaches to deal with
> this yet.

Cheers,
mwh