[pkg-go] RFC: autopkgtest

[Martín Ferrari]

Update: autodep8 and dh-golang have been uploaded, and so we are ready
to start using autopkgtest.

Antonio will whitelist most of our packages in the CI infrastructure to
help bootstrapping, but from now on, we should start adding this field
to debian/control to explicitly enable the automatic tests:

Testsuite: autopkgtest-pkg-go

If you want a package to run other tests instead of the default tests,
you can place a debian/tests/control file. If you want to run extra
tests in addition to the default ones, you can add a
debian/tests/control.autodep8 file, or a autopkgtest target in
debian/rules (this is supported by the script in dh-golang).



On 26/05/17 04:35, Martín Ferrari wrote:
> An status update on autopkgtest for pkg-go.
> 
> I have been working on this since SunCamp, and I think it is ready for
> deployment.
> 
> I have created a much more polished script, and committed it to
> dh-golang. If nobody is against it, I will be uploading it during the
> weekend.
> 
> I have also added golang support to autodep8 (which uses the script in
> dh-golang) and pushed the changes to git master. (Antonio, please let me
> know if you think this change is OK).
> 
> Using these changes, I have already tested all packages that
> build-depend on golang, and I am satisfied that my scripts work for most
> of the cases out of the box, and that the majority of the team's
> packages pass the tests correctly.
> 
> If Antonio agrees, I would like to set up a whitelist for ci.debian.net
> to start testing these soon, so we don't need wait for the next upload
> to enable autopkgtest in 700+ packages.
> 
> 
> The stats from my tests are:
> 
>   694 Tests passing OK
>    33 Tests failing, but maintainer decided to ignore these (FIXME!)
>    16 Failures that seem like real problems.
>    22 Failures because packaging incompatibilites with the script.
>    17 Packages that already have autopkgtest support, or not relevant.
> 
> The names for each package are included in the attached lists.
> 
> 
> On 23/05/17 11:26, Martín Ferrari wrote:
>> Hey all!
>>
>> As I mentioned on IRC, I have been working on getting autopkgtest
>> coverage for all of our packages.
>>
>> I started by writing an script[1] that re-runs the same tests we use
>> during package build automatically. If all packages gets this testing in
>> ci.debian.net, we can automatically detect any regressions due to
>> changes in dependencies, which is IMHO one of our biggest problems nowadays.
>>
>> Later we can add more specialised tests, or re-enable tests that can't
>> run at buildtime because they require networking or root access.
>>
>>
>> To make this available to all packages, the first thing to do is to add
>> golang support to autodep8, which I am preparing.
>>
>> The second thing I need to do is to add the test running scripts to some
>> package we control. It could be an ad-hoc package like
>> pkg-perl-autopkgtest, or it could be integrated into something we
>> already have, like dh-golang.
>>
>> I am currently leaning towards adding this same script to dh-golang,
>> maybe adding detection for a pkgautotest target in debian/rules (which
>> would be run automatically too), and getting a whitelist in
>> ci.debian.net to start getting test reports for all our packages.
>>
>>
>> Ideas, comments?
>>
>>
>> [1]:
>> https://anonscm.debian.org/cgit/pkg-go/packages/prometheus-postgres-exporter.git/tree/debian/tests/build-test
>>
> 
> 


-- 
Martín Ferrari (Tincho)