[Pkg-rust-maintainers] Bug#1054658: rustc: merge src:cargo into src:rustc

[Fabian Grünbichler]

Source: rustc
Severity: wishlist
X-Debbugs-Cc: debian-rust at lists.debian.org

as discussed in the last rust team meeting, we'd like to merge the
packaging of src:rustc and src:cargo. this bug report is filed to have a
place for discussing potential objections and collect input.

a bit of background about the status quo. the rust toolchain consist of
many parts, only some of which are packaged for Debian. the two big
components are cargo (the package manager and build tool) and rustc (the
compiler + stdlib). while some parts of the toolchain are developed
upstream in their own repository, the development of the toolchain
itself happens via a single repository:

https://github.com/rust-lang/rust/

similarly, the official release artifacts are created for all components
from this repository, including binary artifacts and source tarballs,
such as
https://forge.rust-lang.org/infra/other-installation-methods.html#source-code

currently, the packaging structure is like this:

1.) src:rustc

uses a repacked/pruned version of the official toolchain source tarball.
pruning happens via subset of debian/patches and some helper scripts.
components which are not needed for packaging in Debian are removed, to
minimize vendoring.

among other things, an embedded copy of LLVM, the cargo tool and
documentation, non-essential parts of rust-analyzer and other tools
either not packaged or packaged separately are removed.

besides the compiler and std library, src:rustc currently also builds
binary packages for two adjacent tools: clippy and rustfmt, as well as a
binary needed for macro support in rust-analyzer, helpers for
integration into gdb and lldb, and documentation.

clippy, rustfmt and rust-analyzer are all developed as standalone
projects in their own git repositories, but merged into the main rust
toolchain repository and released as one toolchain upstream.

2.) src:cargo

this source package and the corresponding binary packages contain the
cargo tool and some helper scripts specific to Debian. it currently uses
GH tag/release tar balls from the main development repository of cargo.
pruning of the tarball happens via a complicated procedure that attempts
to re-use patches for vendored crates from debcargo-conf (the Debian
Rust team monorepo for packaging regular crates). this procedure is
non-deterministic (it runs `cargo vendor` which can pick up newer
versions if run at a later point in time) and quite involved. as a
result, updating src:cargo usually entails updating a big chunk of
crates packaging via debcargo-conf to reduce the version drift between
the packaged versions which patches we want to reuse, and the versions
picked up by `cargo vendor`.

3.) src:rust-cargo

since there are at least two packages in Debian that employ cargo as
library, cargo is also packaged as a regular library crate via
crates.io. this package is then used as build-dependency for other
packages, like debcargo, the main helper used by the Debian Rust team.


the proposal is to ditch src:cargo in favor of moving the associated
helpers to src:rustc, and no longer patching out cargo-related
components of the toolchain there. as a result, bin:cargo and
bin:cargo-doc would be built from src:rustc, with both parts being built
from the same upstream toolchain release tarball, instead of from two
different "releases". src:rust-cargo would stay as it is now.

pros:

- Ubuntu already made this switch a couple versions back, syncing in
  both ways would be easier[0]
- rustc and cargo would always be packaged in lockstep, instead of cargo
  lagging behind like it has most of the time in the recent past
- the complicated patch syncing step is gone
- one less source package with special vendoring considerations
- packaging the cargo book instead of just rustdoc API documentation
  would be feasible
- bin:cargo would finally have the right version number (right now, it
  has the "eternally unstable" one of the cargo library crate)

cons

- more care needs to be taken when pruning the toolchain release
  tarball, since cargo has more network related vendoring where we don't
  want to bundle C libraries by accident
- effort to switch over packaging (although a lot of the Ubuntu change
  can likely be re-used)


any input/feedback would be appreciated!

0: https://bugs.launchpad.net/ubuntu/+source/rustc/+bug/2020000
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-rust-maintainers/attachments/20231027/d22b2399/attachment.sig>