[Pkg-rust-maintainers] debcargo and dh-cargo ready for production use; sponsor request

Thu Dec 8 01:47:00 UTC 2016

Firstly, thank you for doing all of this! It's a very very impressive effort. I definitely support automating as much of the package creation process as possible!

Josh Triplett:
> [..] For now, you can `cargo install debcargo --root ~/.local`,
> and make sure your PATH includes ~/.local/bin .)
> 

I tried to play with it, but I got an error!

Build failed, waiting for other jobs to finish...
error: failed to compile `debcargo v1.1.0`, intermediate artifacts can be found at `/tmp/cargo-install.1rJ0sZmiJiDM`

Caused by:
  failed to run custom build command for `openssl-sys-extras v0.7.14`
process didn't exit successfully: `/tmp/cargo-install.1rJ0sZmiJiDM/release/build/openssl-sys-extras-5c7e4d8925825f00/build-script-build` (exit code: 101)
--- stdout
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
PROFILE = Some("release")
TARGET = Some("x86_64-unknown-linux-gnu")
debug=false opt-level=3
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = None
HOST = Some("x86_64-unknown-linux-gnu")
TARGET = Some("x86_64-unknown-linux-gnu")
HOST = Some("x86_64-unknown-linux-gnu")
CFLAGS_x86_64-unknown-linux-gnu = None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
running: "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-m64" "-fPIC" "-o" "/tmp/cargo-install.1rJ0sZmiJiDM/release/build/openssl-sys-extras-5c7e4d8925825f00/out/src/openssl_shim.o" "-c" "src/openssl_shim.c"
cargo:warning=src/openssl_shim.c: In function ‘DH_new_from_params’:
cargo:warning=src/openssl_shim.c:132:7: error: dereferencing pointer to incomplete type ‘DH {aka struct dh_st}’
cargo:warning=     dh->p = p;
cargo:warning=       ^~
cargo:warning=src/openssl_shim.c: In function ‘X509_get_extensions_shim’:
cargo:warning=src/openssl_shim.c:143:13: error: dereferencing pointer to incomplete type ‘X509 {aka struct x509_st}’
cargo:warning=     return x->cert_info ? x->cert_info->extensions : NULL;
cargo:warning=             ^~
ExitStatus(ExitStatus(256))

command did not execute successfully, got: exit code: 1

--- stderr
thread 'main' panicked at 'explicit panic', .cargo/registry/src/github.com-1ecc6299db9ec823/gcc-0.3.39/src/lib.rs:984
note: Run with `RUST_BACKTRACE=1` for a backtrace.

# exit code 101

$ sudo apt-cache policy libssl-dev
libssl-dev:
  Installed: 1.1.0c-2

is what I have...

> You can run `debcargo package cratename` to generate a Debian source
> package for a crate from crates.io, ready for review and (in most cases)
> ready for upload.
> 
> [..]
> 
> debcargo and dh-cargo now handle Cargo "features" for crates.  [..]
> 
> [..] I added a --multi option [..]
> 

As discussed on IRC, I'm glad this is now the default.

> [..] debcargo now defaults
> to omitting the binary from a crate that has both, unless you pass
> --bin. [..] You can
> also rename the binary package using --bin-name, to avoid conflicts with
> existing packages.  debcargo will still generate a binary package by
> default for crates that don't also build a library, and for some library
> packages we'll want to pass --bin, such as for future packaging of cargo
> (which includes both a library and the "cargo" binary).
> 

This is probably something that you should prompt the developer for, otherwise they might not notice. That is, if the crate has a binary package then force the developer to supply a --bin-name. Or generate a default one but have them manually confirm it, like how dh_make works.

> As discussed in my previous mail, I've reworked dh-cargo to never invoke
> cargo directly on unpacked .crate sources; [..]
> 
> debcargo now scans through sources for copyright notices and
> automatically includes them in debian/copyright, which eliminates the
> copyright-without-copyright-notice lintian warning (at least for sources
> that have copyright notices at all, which many don't bother with).  Note
> that this does produce some false positives for crates that have license
> files describing third-party licenses that apply to upstream-built
> binary packages that embed libraries but not to binary packages that use
> system libraries; for instance, cargo's LICENSE-THIRD-PARTY lists many
> copyright notices and licenses that wouldn't apply to Cargo packages we
> built via debcargo.  We may need to tweak this over time as we package
> more crates.  (I would rather add piles of heuristics in debcargo than
> tweak any of the generated packaging. :) )
> 

+1 to extra heuristics. Also, as discussed on IRC, you should make it as close to DEP-5 as possible, ideally DEP-5. I would have more specific comments and concrete suggestions on how to do this, but I couldn't run it yet due to the openssl error above.

> debcargo-generated packages generate a description from the upstream
> crate description [..]
> 
> I've submitted a patch to Lintian to handle upstream sources [..]
> 
> I've filed <https://bugs.debian.org/845576> on ftp.debian.org requesting a
> "rust" section, [..]
> 
> Finally, while working on this, I ran into one issue in upstream Cargo,
> for which I've submitted an upstream pull request at
> <https://github.com/rust-lang/cargo/pull/3369>.  [..]
> 

Very thorough, thanks for going through all this effort!

> Note that none of this automation solves the issue of upstream crates
> that embed library sources directly in the crate sources.  [..]
> 
> Many packages will autodetect a system version of the library
> if available. [..]
> 

How does debcargo (or even cargo) declare dependencies on non-rust software? You mentioned on IRC that this is a big hole at the moment, but I wonder if we can begin to patch this from the Debian / debcargo side. Is there a "convention" currently in the rust ecosystem for doing this?

Because of all of these issues, I think it's unavoidable that we will have to manually edit some packages. This means that we should figure out a workflow to make this smooth, but still be able to take advantage of debcargo in the long run.

Something like this:

librust-foo$ ls -la debian/
[.. debcargo generated package, but with extra manual edits ..]

librust-foo$ debcargo update
[.. downloads version from d/changelog, generates packaging for it ..]
[.. downloads latest version, generates packaging for it ..]
[.. diffs the two packaging directories, and outputs the diff and/or tries to apply it to the current tree ..]

Or this:

librust-foo.git$ git status
On branch debcargo-auto

librust-foo.git$ ls -la debian/
[.. original debcargo-generated packaging ..]

librust-foo.git$ git checkout master
librust-foo.git$ ls -la debian/
[.. debcargo generated package, but with extra manual edits ..]

librust-foo.git$ debcargo update
[.. downloads latest version, generates packaging for it, commits it on top of the debcargo branch, tries to merge it back into master ..]
[.. sort of like how `gbp import-orig` works ..]

MERGE CONFLICT, ruh-roh!!

[.. hack hack hack ...]
librust-foo.git$ git add -- [etc etc]
librust-foo.git$ git commit -a
# great success

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git