Bug#870334: pkg-perl-autopkgtest: revisiting smoke prove --recurse

[gregor herrmann]

On Tue, 01 Aug 2017 19:58:55 +0300, Niko Tyni wrote:

> Hm. I agree the goal is to get the same list of tests that are run at
> build time. 

I don't agree 100% I think.
From what I've seen so far when looking at a dozen of the failures we
have:
- same tests in autopkgtest as during build, which fail because of
  paths or missing files; so just the usual stuff but it appears only
  now for t/**/*.t;
- more tests in autopkgtest than during build but perfectly
  reasonable tests where it looks more like an issue that they are
  not run during build (and the typical easy-to-fix-failures);
- and then the group where something weird is in t/ which happens to
  look like a test but is a data file or a (misplaced and not
  environment-variable-protected) author test etc.
  Which is somewhat a case for #870252

> Alternatively, if we're running Makefile.PL anyway (I'm ignoring M::B
> for the moment), this might all be solved if we could just force run
> 'make test' without actually building first. The tests would get run
> with blib/lib etc. in @INC but if those were empty the installed files
> would presumably get used anyway. Unfortunately make doesn't seem to be
> too helpful with this...

Hm, we could also "make" the distribution and rm blib/ before "make
test". Doesn't feel very clean though ...
 
> One more wild idea I had is to patch the build machinery (probably EU::MM
> itself) to record the list of tests it runs in a file. We could then
> install that file in the binary package during the build, and look
> there in the autopkgtest phase. This does feel somewhat intrusive,
> just mentioning it for completeness :)

We had similar ideas as well :)


Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at/ - Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
   `-   
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 963 bytes
Desc: Digital Signature
URL: <http://lists.alioth.debian.org/pipermail/pkg-perl-maintainers/attachments/20170801/3261347b/attachment.sig>