on the topic of using /bin/true as installability autopkgtest

[Paul Gevers]

Dear GIS project maintainers,

It has come to my attention that several of your packages have
autopkgtests (great) in the form of:
Test-Command: /bin/true
Depends: @

I want to draw to your attention the following quote [1] from the
release team:
'''
For this proposal to make sense, all deployed autopkgtests must
actually test the package involved to some extent.  We trust it will
not be necessary to establish a technical solution for this part.
'''

I have verified with the release team and they stand by that quote and
they consider this kind of installability testing via autopkgtest not
"testing the package involved to some extent" if it is the _only_ test.
Also, installability is better tested with piuparts [2] (which, on
failure, will fully block migration). The reason why this is undesirable
is that packages with a passing autopkgtest (and no regressions
otherwise) receive a reduced migration age. The release team and I
consider installability not enough reason to grant that reduction.

Please stop adding this type of autopkgtest to your packages if you
don't also add more substantial tests. And please start updating your
existing tests. I have just filed lintian bug #905030 to have a check
for this.

If you use these autopkgtest results in any infrastructure and it is
difficult or much work to move away from it, I'd like to draw your
attention to a recent enhancement of autopkgtest: the skippable
restriction [3]. If you let your test always fail with exit code 77 (so
replace /bin/true with exit 77) you could check yourself for the exit
state of autopkgtest, while the test doesn't get the age reduction (two
patches [4][5] are still required for the Debian infrastructure to
handle this OK, but I can manually help in the mean time).

Paul

[1] https://lists.debian.org/debian-devel-announce/2013/08/msg00006.html
[2] https://piuparts.debian.org/
[3]
https://salsa.debian.org/ci-team/autopkgtest/raw/master/doc/README.package-tests.rst
[4] https://bugs.debian.org/901804
[5] https://bugs.debian.org/901846

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/pkg-grass-devel/attachments/20180730/e32d7773/attachment.sig>