Bug#905885: diffoscope: skipping tests on ci.debian.net is perhaps wrong?
mattia at debian.org
Mon Sep 3 15:42:20 BST 2018
Control: allow to override @skip_unless_tools_exist during tests and fail the test if the tool is missing
On Sun, Aug 12, 2018 at 12:03:50AM +1000, Stuart Prescott wrote:
> *also* diffoscope recommends apktool but tests on ci.d.n do not fail (but
> should, I think)
what you say here it's half true: the autopkgtest has Depends: all the
recommends that we have. What is happening is that ci.d.n/autopkgtest
(no clue what piece of the puzzle is responsable for this) is
downloading the test dependencies from unstable if they are not
available on testing, silently.
I believe you may want to get in touch with the CI team for this, I
don't think there is much a package can do here.
> > > Perhaps @tool_required could be extended with an "all tools required"
> > > mode such that no tests are skipped or missing tools are a test failure
> > > (or error). An environment variable would work for setting that mode
> > > and that mode used at package build time and in autopkgtest.
> > (One difficulty with "just" extending @tool_required is that sometimes
> > we really do want to skip the tests as they require a newer/older
> > version of a particular tool in order that we can correctly match up
> > the output.)
That's an interesting idea, and I'm retitling this bug to do this.
Though what we would need to twak is the @skip_unless_tools_exist
decorator we use in the tests.
Also, we have comparators (and tests) for tools that Debian doesn't
ship, so if we add a flag to make @skip_unless_tools_exist fail a test
instead of skipping it, we would also need to provide a mechanism to
"override the override" and have it really skip those :)
I'm not exactly sure how we should handle the various other
@skip_unless_tool_is_* and friends though...
GPG Key: 66AE 2B4A FCCF 3F52 DA18 4D18 4B04 3FCD B944 4540 .''`.
more about me: https://mapreri.org : :' :
Launchpad user: https://launchpad.net/~mapreri `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia `-
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 833 bytes
Desc: not available
More information about the Reproducible-builds