[Pkg-rust-maintainers] Packaging advice/help wanted for anki, which has just started using some Rust

[Wolfgang Silbermayr]

Hi Julian,

On 1/21/20 11:00 AM, Julian Gilbey wrote:
> I maintain the Python-based package "anki", which has been almost no
> problem up to now.  But upstream have just started incorporating some
> Rust code into it, and I've never used Rust myself.  The build
> mechanism they have written involves downloading and compiling Rust
> modules (I think that's what it's doing) using rustup.

This is clearly not something we want to have inside the build process.
rustup is a tool we might want to package in the future, but just for
users who want to manage user installations of their rust toolbox, most
certainly not for installing a toolchain during package build process.

> Here's a piece
> of the Makefile as an example:
> 
> # nightly currently required for ignoring files in rustfmt.toml
> RUST_TOOLCHAIN := $(shell cat rust-toolchain)
> 
> .build/tools: requirements.txt rust-toolchain
> 	pip install -r requirements.txt
> 	rustup toolchain install $(RUST_TOOLCHAIN)
> 	rustup component add rustfmt-preview --toolchain $(RUST_TOOLCHAIN)
> 	rustup component add clippy-preview --toolchain $(RUST_TOOLCHAIN)
> 	@touch $@
> 
> .build/check: build
> 	cargo fmt -- --check
> 	cargo clippy -- -D warnings
> 	@touch $@

The snippets listed here should not be necessary for the build process.
The `toolchain install` subcommand installs a specific toolchain into
the user directory (in this case pinned to a nightly given by the
contents of the rust-toolchain file). The `component add` subcommand
adds the rustfmt and clippy tools into the previously installed
toolchain. These tools are helpful for development of rust code, but not
necessary for the build process:
* cargo fmt / rustfmt formats the code, or if called with --check as
  here, informs about code not formatted according to the style defined
  in `rustfmt.toml`.
* cargo clippy is a linting tool for rust code.

So I assume the package can be built without the whole rustup foo.

> (This is a little weird, as there's no rule to make rust-toolchain.
> Perhaps an error in the Makefile?)

Maybe they reference the `rust-toolchain` file in the same directory? So
the `.build/tools` target depends on changes inside that file.

> So any advice on how to handle this?  There is no "rustup" package in
> Debian (presumably for a good reason), but I have found the list of
> available rustup components here:
> https://rust-lang.github.io/rustup-components-history/
> Is there any way to do this process within the existing Debian Rust
> infrastructure?

We might get significant parts of that done with what we have, but
there's some new things as well.

> Also, is there a Debian Rust Policy?  I haven't been able to find one
> on a quick scan.

There is some general information in [0] and a policy in [1]. It has
been some time since the policy page got updated, so some details might
not be completely up-to-date.

A quick summary of how we package Rust code:
Rust projects are called crates, they can come in different flavors,
explained in [2]. For now, the debcargo tool can only handle crates
published to the crates.io platform, and only of the "lib" and "bin"
crate types. Initial support for packaging crates not published there is
in the works, but it might take some time until it is ready [3]. The
"lib" crates we package, contain a copy of the source code which gets
used by the `cargo` tool during the build process, by telling it the
path to the local cargo registry containing the source code. The other
rust crate packages then build-depend on the required lib packages, so
that these can be found in the registry. A "bin" crate usually yields a
static binary executable. This will probably be equivalent for "cdylib"
crates which will contain a .so file.

A quick look at the anki git repo shows the `rslib` subdirectory
containing a project of the "lib" crate type. This lib gets used by the
`rspy` crate which is of the "cdylib" crate type. So the `rspy` crate
needs to be built with the `rslib` crate available (either packaged with
e.g. debcargo once the feature is available, or "vendored"). Also the
other dependencies as listed in both `Cargo.toml` files need to be
present. Note that the crate version numbers of dependencies are in
semver format, so a `mycrate = "0.1.2"` means any version 0.1 greater
than 0.1.2, and a `mycrate = "6.4.2" means any version 6.x greater than
6.4.2. A successful build will result in a
`rspy/target/release/libankirspy.so` shared object file which you will
have to install in the correct path. `resvg`, a "cdylib" type crate has
been packaged. I haven't taken a close look at that package, but maybe
you find some inspiration there.

Some of the required dependencies are already packaged while others are
not. We use the `cargo-debstatus` tool [4] for analyzing the dependency
tree branches that need to be packaged. It's however not packaged for
Debian yet, so it needs to be installed by hand for now (cargo install
cargo-debstatus; installs into ~/.cargo/bin).

Please note that I don't have too much insight into how python library
packages are structured, and how the FFI into shared objects works. Also
my knowledge regarding "cdylib" rust crates is very limited for now.
None the less I hope you got some basic idea of the general structure at
hand.

Best regards,
Wolfgang.

--

[0] https://wiki.debian.org/Teams/RustPackaging
[1] https://wiki.debian.org/Teams/RustPackaging/Policy
[2] https://doc.rust-lang.org/reference/linkage.html
[3] https://salsa.debian.org/rust-team/debcargo/merge_requests/22
[4] https://crates.io/crates/cargo-debstatus