[Pkg-puppet-devel] Presenting dh-puppet

[Jérôme Charaoui]

Le 2024-03-06 à 10 h 27, Antoine Beaupré a écrit :
>> This dh-puppet project is only a beginning. If there's enthusiasm for
>> it, I'm also planning to work on autodep8 integration. This is where it
>> gets interesting, even for those of you who don't use module packages at
>> all. Having autopkgtests on these packages will be a great help in
>> identifying many Puppet module regressions in testing/sid. The issues
>> identified (often trivial, like package name changes) will help us help
>> module upstreams keep their modules compatible with the latest Debian,
>> and make it easier and more convenient to puppetize our own testing/sid
>> hosts.
> 
> autopkgtest would be *awesome*! I have slight concerns about the
> direction you were taking with that however; on IRC, you seemed to be
> saying you want to write manifests specific for the autopkgtest
> suite... For me, we should just run the upstream test suite
> directly.
> 
> I understand this might be more complicated to implement or might just
> fail, so you're leaning towards writing a simple "smoke test" with a
> `.pp` manifest, and I think that's a good short-term fix. But even if we
> do go *that* way, I would much rather see those in the upstream
> `examples/` directory, where it can (eventually!) be submitted upstream,
> than having a permanent divergence in the `debian/tests/` dir.

So I looked into this a little more closely, and the module upstreams 
already have the kinds of tests we want for autopkgtests: the 
"acceptance" tests located under "spec/acceptance". Those are intended 
to apply manifests, check for idempotency, and verify things like are 
services running, ports open, etc. Exactly what we want to autopkgtests.

The problem is they seem to have widely varying dependencies and most or 
all of them aren't in Debian. For example, puppetlabs modules rely on 
puppet_litmus [0] which handles not only running the tests but also 
provisioning and managing whole testing environments (eg. containers and 
virtual machines), which is something we don't need at all in the 
context of autopkgtests.

Most voxpupuli module acceptance testsuites, on the other hand, depend 
on voxpupuli-acceptance, which is completely unrelated to puppet_litmus, 
and instead depends on beaker, which also is aimed at provisioning 
testing environments. There used to be a puppet-beaker package in Debian 
but it's been unmaintained and removed from the archive.

Furthermore, those tests are designed to run with the Puppetlabs 
packages (example [3]), so at least some of our module packages would 
need patches to the testsuite or they wouldn't work at all. So, yes, 
"complicated to implement" an accurate description!

In terms of submitting "smoke test"-type manifests under the examples 
directory in each module, I'm not sure this is feasible either. The 
content under that directory is intended as documentation, not tests, so 
it would likely be rejected solely on that basis.

It's also worth mentioning that working with the puppet module upstreams 
is already difficult, and a number of patches we've submitted for Debian 
*12* support are still lingering in pull requests without review. I 
would much rather leverage what little attention we can get from these 
(often very busy) upstreams to push module fixes instead of working on a 
new kind of "ecosystem standard" proposal and spending time trying to 
convince voxpupuli and friends to adopt it.

[0] https://github.com/puppetlabs/puppet_litmus
[1] https://github.com/voxpupuli/voxpupuli-acceptance
[2] 
https://github.com/voxpupuli/puppet-redis/blob/master/spec/acceptance/redis_cli_task_spec.rb