Bug#922264: pkg-perl-autopkgtest: use "skippable" and "superficial" restrictions
gregor herrmann
gregoa at debian.org
Fri Feb 15 16:57:17 GMT 2019
On Wed, 13 Feb 2019 21:23:00 +0100, Xavier Guimard wrote:
> Some suggestions for pkg-js-autopkgtest based on pkg-js-autopkgtest
> discussion with autodep8 maintainers:
> - tests skipped should return a 77 exit code and all tests marked as
> "Restrictions: skippable". It avoids to consider that a test succeeds
> if maintainer skipped it, but needs a merge request to autodep8. See
> https://salsa.debian.org/ci-team/autodep8/blob/master/support/nodejs/generate
> (changed by MR !11)
> - runtime-deps* tests should be tagged as "Restrictions: superficial"
> since these tests don't really test package features but just Perl
> syntax
>
> Then with this 2 changes, if "build-deps.d" is skipped, success won't
> give the benefit of 3-days-reduce.
Thanks for your work and the pull request [0]!
Some thoughts and more questions:
For the skippable part:
- If I understand this correctly (from your text above and the spec
[1]) then a skipped syntax.t and use.t would also lead to losing
the benefit of faster migration? Do we want this?
Or does it just have no influence?
- As for the implementation in [0]:
not sure if the "exit 0" in smoke is correct
- What about the skipped tests within use.t and syntax.t? Should they
or some of them also exit 77?
For the superficial part:
Hm, yeah, use.t and syntax.t don't test that everything in the
package is fully functional; still, this "superficial" feels a bit
weird. But probably it's correct according to [1].
In general I still don't have the full picture of what benefits and
penalties for testing migration will result from which combination of
the changes under which circumstances.
Cheers,
gregor
[0] https://salsa.debian.org/perl-team/modules/packages/pkg-perl-tools/merge_requests/2
[1] https://salsa.debian.org/ci-team/autopkgtest/raw/master/doc/README.package-tests.rst
--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`- NP: Cat Power: The Greatest
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 963 bytes
Desc: Digital Signature
URL: <http://alioth-lists.debian.net/pipermail/pkg-perl-maintainers/attachments/20190215/00f9ed36/attachment.sig>
More information about the pkg-perl-maintainers
mailing list