[Pkg-rust-maintainers] Bug#985741: debcargo 2.4.4 adds --no-default-features to every feature-specific test in debian/tests/control

[Daniel Kahn Gillmor]

Package: debcargo
Version: 2.4.4
Control: affects -1 src:rust-sequoia-openpgp

in rust-sequoia-openpgp 1.0.0-1 (generated with debcargo 2.4.3), we see
this example test of the "compression-deflate" feature:

    Test-Command: /usr/share/cargo/bin/cargo-auto-test sequoia-openpgp 1.0.0 --all-targets --features compression-deflate
    Features: test-name=librust-sequoia-openpgp+compression-deflate-dev

But in rust-sequoia-openpgp 1.1.0-1 (generated with debcargo 2.4.4, with
no upstream changes to compression-related stuff), we have this test of
the "compression-deflate" feature:

    Test-Command: /usr/share/cargo/bin/cargo-auto-test sequoia-openpgp 1.1.0 --all-targets --no-default-features --features compression-deflate
    Features: test-name=librust-sequoia-openpgp+compression-deflate-dev:compression-deflate

The test is named slightly differently, but the biggest change is that
it now includes the --no-default-features flag.

This is a subtle but important difference.

Here is a full set of feature-related tests that we can imagine running:

 a) no features enabled at all

 b) all features enabled together

 c) only default features, enabled together

 d) one test per non-default feature, with the default features also enabled

 e) one test per feature, with no other features enabled.

This change from 2.4.3 to 2.4.4 effectively swaps out the tests in (d)
for the tests in (e).

(fwiw, i can also imagine some sort of horrible combinatorial explosion
where we try to test every possible combination of features; i won't get
into that here)

For rust-sequoia-openpgp, where at least one crypto-backend feature must
be enabled (and nettle is the only crypto-backend feature on debian
systems), and the default featureset does include crypto-nettle, all the
tests in (d) should succeed, while most of the tests in (e) are
likely to fail.

I'd prefer to test the combinations of features in rust-sequoia-openpgp
that feel most useful to me (and to minimize the number of tests that
we need to mark as flakey) so i'd prefer to cover (d) instead of (e).

I can see a handful of different possible ways to resolve this:

 0) revert to the 2.4.3 behavior, by dropping --no-default-features from
    the feature-specific tests.

 1) introduce a variable debcargo.toml that lets the maintainer choose
    between (d) and (e), e.g., "test_with_default_features" as a variable
    for the packages.lib section. 

 2) generate even more tests per package, including both (d) and (e)
    automatically (and distinctly) as long as there are some default
    features.

 3) stick with the 2.4.4 behavior, but clearly document that these extra
    tests will be run with --no-default-features.

Regards,

        --dkg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-rust-maintainers/attachments/20210322/cbaa239a/attachment.sig>