[Pkg-samba-maint] On the selftest in debian work

[Andrew Bartlett]

On Fri, 2016-05-06 at 15:57 +0200, Mathieu Parent wrote:
> 2016-05-05 20:44 GMT+02:00 Jelmer Vernooij <jelmer at jelmer.uk>:
> > On Fri, May 06, 2016 at 06:33:45AM +1200, Andrew Bartlett wrote:
> [...]
> > However, the work to get to get the test runs stable should really
> > happen
> > upstream and not in the Debian packaging. This is not specific to
> > Debian.
> > The upstream testsuite shouldn't fail regularly. Until it is stable
> > there,
> > I don't see a point in trying to get it to run on Debian.
> 
> I agree. but currenly the quicktest only fails on the raw.write
> tests,
> under 32-bit only.
> 
> I've removed this test (actually 3 tests) on my local repo, currently
> rebuilding on i386.
> 
> If everything goes well, I'll upload this. Let's see if the flappy
> test is still here (it was on amd64 reproducible buildd).
> 
> I'm sorry that my uploads have stopped samba 4.4 to go to testing.
> Will fix this soon.

I'm really sorry, but we need to first work on this upstream, then in
experimental.  There are still too many flapping tests to do this
reliably in something like Debian, certainly for unstable.

Have a read of 
https://lists.samba.org/archive/samba-technical/2016-May/113894.html fo
r the sorry state of affairs.

We first need to get this reliable upstream, then in experimental, as
Jelmer and I have discussed.  It is a very worthwhile task, but not a
small one.

Because of the nature of the Samba flapping tests, you need to have a
large sample, not just if a build works once.

To be clear, I'm currently running 10 builds in large VMs on the
Catalyst cloud to try and get a reasonable statistical overview of the
tests for a bug I'm fixing at work.  If the builds run on slow or busy
hosts, we can expect spurious failures.  

Sorry,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba