Bug#870334: pkg-perl-autopkgtest: revisiting smoke prove --recurse

[Niko Tyni]

Package: pkg-perl-autopkgtest
Version: 0.37

Since 0.37, we're running 'prove --recurse' by default in the smoke
test. This has resulted in ~50 regressions according to ci.d.n:
there are packages ship .t files under t/ subdirectories (or starting
with a dot) that are not run during builds and are either broken
or not intended to be run at all.  Fixing these by adding t/*.t in
debian/tests/pkg-perl/smoke-tests gets boring quite quickly, though it's
certainly doable.

All these packages seem to be relying on the ExtUtils::MakeMaker default
of TEST_FILES => t/*.t. I'm wondering if we should try to parse that
out of Makefile.PL during autopkgtest instead of always recursing,
though that does seem somewhat error prone.

FWIW, Module::Build seems to have a related 'recursive_test_files'
option that is off by default (see Module::Build::API(3pm)).

It might help a bit to collect some statistics about the number of
packages with *.t files in subdirectories, and whether those have
TEST_FILES specified in Makefile.PL. Currently I have no idea if the
failing packages are just a small minority of the affected ones, or if
almost all of them are failing now.
-- 
Niko Tyni   ntyni at debian.org