[Pkg-rust-maintainers] Upstream has changed the bootstrap process

Tue Jul 5 16:47:32 UTC 2016

Sylvestre Ledru:
> Le 03/07/2016 à 11:30, Luca BRUNO a écrit :
>> On Monday 27 June 2016 20:33:06 Ximin Luo wrote:
>>> Any of you guys coming to DebConf? If so, we could work on this.
>>>
>>> If not, I could even work on this by myself, but I only have a semi-idea on
>>> what I'm doing, so any more specific instructions would be appreciated. :)
>> Unfortunately not, but I'll be available via IRC / hangouts if you have doubts 
>> to discuss.
> Just like Luca...

I'm making some good progress, here:

https://anonscm.debian.org/cgit/pkg-rust/rust.git/commit/?h=_volatile_next

It's not yet working fully, but it's "doing the right thing". Here are the problems I'm running into, building using the debian/ from the above link, but against the rust upstream *master* branch (from github).

## Building with "dlstage0" profile (for bootstrapping)

This uses the orig-dl tarball, updated for their new bootstrap process. As Angus says, this will be for bootstrapping a new arch only, and ideally we would just build-dep on $release-1 (see next section). However, I'm running into a weird error:

$ DEB_BUILD_PROFILES=dlstage0 debian/build
[..]
CFG_LLVM_LINKAGE_FILE=/home/infinity0/ext/git/rust/x86_64-unknown-linux-gnu/rt/llvmdeps.rs LD_LIBRARY_PATH=/home/infinity0/ext/git/rust/x86_64-unknown-linux-gnu/stage0/lib:/usr/lib/llvm-3.7/lib:$LD_LIBRARY_PATH   x86_64-unknown-linux-gnu/stage0/bin/rustc --cfg stage0 -C link-args=-Wl,-z,relro -O --cfg rtopt -C rpath -C prefer-dynamic -C no-stack-check --target=x86_64-unknown-linux-gnu   -L "x86_64-unknown-linux-gnu/rt" -L native="/usr/lib/llvm-3.7/lib"     --out-dir x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib -C extra-filename=-39b92f95 -C metadata=39b92f95 src/librustc_trans/lib.rs
src/librustc_trans/back/link.rs:843:8: 843:13 error: unresolved name `dylib` [E0425]
src/librustc_trans/back/link.rs:843     if dylib && t.options.linker_is_gnu {
                                           ^~~~~
src/librustc_trans/back/link.rs:843:8: 843:13 help: run `rustc --explain E0425` to see a detailed explanation
error: aborting due to previous error
/home/infinity0/ext/git/rust/mk/target.mk:212: recipe for target 'x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.rustc_trans' failed
make[2]: *** [x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib/stamp.rustc_trans] Error 101
make[2]: Leaving directory '/home/infinity0/ext/git/rust'
dh_auto_build: make -j1 all VERBOSE=1 returned exit code 2
debian/rules:94: recipe for target 'override_dh_auto_build-arch' failed
make[1]: *** [override_dh_auto_build-arch] Error 2
make[1]: Leaving directory '/home/infinity0/ext/git/rust'
debian/rules:76: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2
debuild: fatal error at line 1376:
dpkg-buildpackage -rfakeroot -D -us -uc -b failed
29

## Building with previous rustc Debian package

This now correctly uses the system rustc (not yet locked to the previous version, I have a TODO for this). However after a while it fails because of:

$ debian/build
[..]
src/libcore/lib.rs:64:1: 64:37 error: #[feature] may not be used on the stable release channel
[..]

Upstream has their own way of solving this, described below, but I'm having a hard time understanding it properly. Any help here would be appreciated:

commit 02538d463a350f5c3658f7aabefca16eb599d31c
Author: Alex Crichton <alex at alexcrichton.com>
Date:   Wed Apr 13 11:18:35 2016 -0700

    mk: Bootstrap from stable instead of snapshots

    This commit removes all infrastructure from the repository for our so-called
    snapshots to instead bootstrap the compiler from stable releases. Bootstrapping
    from a previously stable release is a long-desired feature of distros because
    they're not fans of downloading binary stage0 blobs from us. Additionally, this
    makes our own CI easier as we can decommission all of the snapshot builders and
    start having a regular cadence to when we update the stage0 compiler.

    A new `src/etc/get-stage0.py` script was added which shares some code with
    `src/bootstrap/bootstrap.py` to read a new file, `src/stage0.txt`, which lists
    the current stage0 compiler as well as cargo that we bootstrap from. This script
    will download the relevant `rustc` package an unpack it into `$target/stage0` as
    we do today.

    One problem of bootstrapping from stable releases is that we're not able to
    compile unstable code (e.g. all the `#![feature]` directives in libcore/libstd).
    To overcome this we employ two strategies:

    * The bootstrap key of the previous compiler is hardcoded into `src/stage0.txt`
      (enabled as a result of #32731) and exported by the build system. This enables
      nightly features in the compiler we download.
    * The standard library and compiler are pinned to a specific stage0, which
      doesn't change, so we're guaranteed that we'll continue compiling as we start
      from a known fixed source.

    The process for making a release will also need to be tweaked now to continue to
    cadence of bootstrapping from the previous release. This process looks like:

    1. Merge `beta` to `stable`
    2. Produce a new stable compiler.
    3. Change `master` to bootstrap from this new stable compiler.
    4. Merge `master` to `beta`
    5. Produce a new beta compiler
    6. Change `master` to bootstrap from this new beta compiler.

    Step 3 above should involve very few changes as `master` was previously
    bootstrapping from `beta` which is the same as `stable` at that point in time.
    Step 6, however, is where we benefit from removing lots of `#[cfg(stage0)]` and
    get to use new features. This also shouldn't slow the release too much as steps
    1-5 requires little work other than waiting and step 6 just needs to happen at
    some point during a release cycle, it's not time sensitive.

    Closes #29555
    Closes #29557

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
https://github.com/infinity0/pubkeys.git