Bug#870334: pkg-perl-autopkgtest: revisiting smoke prove --recurse

[gregor herrmann]

On Wed, 02 Aug 2017 11:10:25 +0300, Niko Tyni wrote:

> > - more tests in autopkgtest than during build but perfectly
> >   reasonable tests where it looks more like an issue that they are
> >   not run during build (and the typical easy-to-fix-failures);
> If the issue here is that they should be run during build too, the "right"
> fix here as well seems to be unifying the list of build and runtime tests?

Right, just that the build time test side has to be fixed upstream
(with or without patches from us).
 
> > - and then the group where something weird is in t/ which happens to
> >   look like a test but is a data file or a (misplaced and not
> >   environment-variable-protected) author test etc.
> >   Which is somewhat a case for #870252
> Sure, we should filter the more common tests of this kind out
> of the runtime check list.

Ack, and this should bring us a long way.
(I saw quite a few t/author/ and t/00-compile/ today.)
 
> I still maintain that if we can automatically determine the list of tests
> that get run during build time, we should base the list of runtime tests
> on that rather than recursing blindly. Of course, if this determination
> is too hard, it may be that recursing is good enough.
> Do we agree on that?

More or less :)
Yes, it makes total sense in theory; I just also happen to like the
idea that we are currently running as many tests as possible.

But yeah, if a realiable implementation of "find out build tests and
run them during autopkgtest" is possible then it probably makes
sense. And ...
 
> The 'make -n -p | grep TEST_FILES' thing seems most promising to me so far
> (even if it's for EU::MM only.)
> 
> I just noticed that the Build files generated by Module::Build have a
> 'retest' option that looks promising:

... since so seem to have ideas for the implementation, by all means
please go ahead :)
 

Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at/ - Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
   `-   
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 963 bytes
Desc: Digital Signature
URL: <http://lists.alioth.debian.org/pipermail/pkg-perl-maintainers/attachments/20170802/94195db9/attachment.sig>