[Pkg-rust-maintainers] Packaging software written in Rust?

Tue May 24 03:58:13 UTC 2016

I definitely found out the hard way that cargo doesn't like to work
without a network, when I tried to hack on some Rust code on an
airplane.

Ignoring Cargo.lock files makes sense.  They have some information of
value, but we can't usefully use any of it.

I agree that we need co-installability of libraries to some extent.  At
least in theory, Rust libraries should use semantic versions, so we
should only need co-installable packages of different major versions; of
course, theory and practice may vary there.

I don't think the license file issue really matters; explicit metadata
in Cargo.toml should suffice. Ditto for only providing the sources via
crates.io; that seems like an acceptable upstream source (as long as the
issues with checksums and verification get sorted out).

I really like your plan of just packaging libraries as source code for
now.  If we can get it to work, it seems likely to produce sensible
results far sooner than getting a binary package ecosystem to work; it
also avoids needing hundreds of bin-NMUs after every rustc upload.  And
the source approach also allows the final program to specify cargo
feature flags for its dependencies; a pre-compiled binary library
wouldn't support that.

I do indeed have a non-hypothetical use case for Rust application
packaging.  I plan to publish a program for production use in the next
few weeks, built via Rust, that I'd like to get packaged (in a properly
policy-compliant way) by various distributions.  I'll post a mail here
when I get it released.

Do you (or any of the other Debian Rust folks) plan to attend RustConf?
Have you considered submitting a talk/BoF proposal on distribution
packaging?

- Josh Triplett

On Tue, May 24, 2016 at 01:28:51AM +0000, Angus Lees wrote:
> I think you've precisely summarised the plans - the only extra bit I'd add
> is ensuring that we correctly add Built-Using fields to the output
> (application) .debs.  Afaik none of this is written up anywhere better than
> the occasional post to this mailing list.  And of course none of the
> tooling actually exists.
> 
> There's been a few early attempts (by me and others) at making cargo do
> what we need, and writing a dh_cargo.  My efforts never got far enough to
> work, and haven't kept up with recent changes to cargo.   I did get far
> enough <https://github.com/anguslees/cargo-debmake> to conclude that adding
> an actual cargo subcommand ("cargo-debbuild") in Rust was feasible and
> reusing cargo libraries was probably of greater benefit than reusing
> (python/perl) debhelper libraries.  Ymmv of course.
> 
> Some minor complications to be aware of:
> - It's awkward to get cargo to not use the network.  You can provide it
> with a complete copy of the crates.io metadata in the exact same directory
> tree layout; there are attempts like
> https://github.com/rust-lang/cargo/pull/2361 which also assume they're
> mirroring an identical subset of crates.io; but I don't think I've seen a
> solution here that makes me immediately think "aha, good."
> - As far as I can work out, the correct thing for us to do as a distro is
> to ignore any upstream Cargo.lock, and provide our own integration
> testing/assurance/repeatability.
> - It's important that multiple versions of Rust build-deps are
> co-installable, since larger projects tend to have transitive dependencies
> on more than one version of common libraries (for better or worse).
> - Many Rust crates have no sensible point to retrieve the source other than
> through crates.io.  In particular, many upstream git repos don't reliably
> have a way to link git revisions to crates.io releases.
> - The only mention of a license for crates is often just the single field
> in the crates.io metadata.
> - A number of Rust *-sys library packages include vendored copies of C
> libraries.  The good news is that all of these that I looked at found+used
> the local system library via pkg-config just fine, so we can strip the
> upstream sources from our packaging and avoid having to audit licenses, etc.
> 
> To avoid you websearching, here's a selection of previous public
> discussions on the topic: (I think they're all my posts, but that's just
> because I could remember what to search for to find those ;)
> - (Early post):
> https://internals.rust-lang.org/t/how-to-distro-package-a-cargo-using-rust-executable/1785
> - (Me explaining the plan again to upstreams):
> https://internals.rust-lang.org/t/perfecting-rust-packaging/2623/57
> <https://internals.rust-lang.org/t/perfecting-rust-packaging/2623/57?u=gus>
> - (A summary of some early hacks):
> https://internals.rust-lang.org/t/perfecting-rust-packaging-the-plan/2767/26
> <https://internals.rust-lang.org/t/perfecting-rust-packaging-the-plan/2767/26?u=gus>
> 
> I'd like to resume some of this, if you have a non-hypothetical need for
> any of it (other than cargo itself).  It was hard to maintain enthusiasm
> when I felt cargo upstream had zero empathy with our use-case :/
> 
>  - Gus
> 
> On Tue, 24 May 2016 at 01:25 Josh Triplett <josh at joshtriplett.org> wrote:
> 
> > [Please CC me on replies.]
> >
> > Do we have any guidelines on packaging software written in Rust, and in
> > particular software built using Cargo?
> >
> > Software packaged for Debian can't let Cargo pull down packages from the
> > network.  Given that, do we have an established procedure to package
> > Rust library crates (from crates.io or otherwise), and then to point
> > Cargo exclusively at the packaged crates?  Preferably in a way that only
> > involves passing options to Cargo, without making any changes to
> > Cargo.toml?
> >
> > Rust doesn't have a stable library ABI ("yet", I hope), so I don't think
> > it makes sense yet to package shared Rust libraries; however, I think we
> > can create -dev packages for crates, and then build-depend on those.  (A
> > simple script, perhaps packaged as a cargo helper command, could
> > automatically translate the dependencies in Cargo.toml into
> > Build-Depends suitable to update debian/control.)  We'll then need to
> > rebuild dependent packages whenever we update to a new version of rustc,
> > or potentially whenever their dependencies change; the latter depends on
> > cargo's handling of pre-compiled library dependencies, as normally cargo
> > rebuilds the entire dependency tree rooted at a binary to keep its
> > dependencies consistent, and we may need to maintain that property
> > ourselves within the Debian archive.  The current transition tracking
> > mechanism (https://release.debian.org/transitions/) should handle that;
> > Rust will likely end up on the list of "Permanent trackers" alongside
> > Haskell and OCaml.
> >
> > In terms of future automation, we'll want to build a dh_cargo at some
> > point, and it can just pass a standard set of options and paths to
> > cargo.  Depending on the handling of library dependencies, we may also
> > want to provide a substvar for debian/control, so that library -dev
> > packages can have automatic Depends on their dependencies.
> >
> > - Josh Triplett
> >
> > _______________________________________________
> > Pkg-rust-maintainers mailing list
> > Pkg-rust-maintainers at lists.alioth.debian.org
> > https://lists.alioth.debian.org/mailman/listinfo/pkg-rust-maintainers
> >