[Pkg-openssl-devel] autopkgtest on openssl

[Kurt Roeckx]

On Sat, May 19, 2018 at 12:51:42PM +0200, Sebastian Andrzej Siewior wrote:
> Hi,
> 
> openssl has a migration time of 15 days due to missing autopkgtest which
> adds +10 days.

It's my understanding that the +10 days is because all reverse
depedencies don't say pass (yet), it's not because we don't have
any test ourself.

> Any idea what would make sense?
> I was thinking about some `openssl' commands like `enc' or `x509' for
> cert signing, verification and maybe expire time.
> That is probably a subset of what openssl's testsuite does but it does
> not run against the *installed* version of binary + libs. And we should
> not compile it (the whole source packag). Some tests however require
> compiled binaries (there are test/*.c files which are invoked as part of
> the testsuite).
> So here I am wondering if we should ping upstream if they would help to
> sort that out (so that we can compile only the testsuite and run it
> against the installed binaries/libs) or if we should do some minimal
> `openssl' binary tests.

It seems very unlikely to me that between building and some time
later openssl started to behave differently. About the only thing
that can have an effect on that is libc and the kernel. To test
that they broke something, it's much easier to just compile it
again, and as far as I know, we do get compiled on a regular basis.

I think it's much more useful to have good test coverage in the
reverse dependencies.

That said, it can never hurt to also have autopkgtest, it was just
not really a priority for me. You can always ask upstream to make
it possible to test agains the installed version.


Kurt