Bug#922264: pkg-perl-autopkgtest: use "skippable" and "superficial" restrictions

[Niko Tyni]

On Fri, Feb 15, 2019 at 08:55:16PM +0100, gregor herrmann wrote:

> > > - As for the implementation in [0]:
> > >   not sure if the "exit 0" in smoke is correct
> 
> This still confuses me.
> Shouldn't it "exit $?" or just nothing (line 174)?

It's a "set -e" script so a failure from test.pl should stop processing
before the 'exit 0'. But yeah, the flow could be a bit clearer.

> > > - What about the skipped tests within use.t and syntax.t? Should they
> > >   or some of them also exit 77?
> > runner do it for them. 
> 
> Well, only partially. First of all, runners can run more than one
> test in a subdirectory (even if we currently only have 3 files in 4
> subdirectories, with 3 times 1 and 1 time zero), second, there are
> several places in syntax.t and use.t were all or parts of the tests
> are skipped. -- But:
> 
> > I didn't modify them if all is skipped as it has
> > no effect on a test marked as "superficial": 0 or 77 gives the same
> > result: no benefit, no penalty
> 
> … this "no benefit, no penalty" makes it indeed kind of moot :)

Given we need to mark the superficial tests as skippable (due to
the runner change), it would seem cleaner to me to make them signal
properly also the other cases of skipping. But I can see it doesn't
matter that much.

> (Random note, so we don't forget it: There are a few adjusted copies
> of the autodep8 file in various packages as debian/tests/control
> which should also be adjusted, at least in git for the next upload.)

AIUI these changes only affect packages without proper 'smoke'
tests. When 'smoke' is working correctly, neither the 'superficial'
nor the 'skippable' part really matters.

So adjusting those individual copies doesn't seem much of a priority
unless there are some where 'smoke' is skipped (or left out).
-- 
Niko