[Pkg-samba-maint] On the selftest in debian work

[Mathieu Parent]

2016-05-06 23:04 GMT+02:00 Andrew Bartlett <abartlet at samba.org>:
> On Fri, 2016-05-06 at 15:57 +0200, Mathieu Parent wrote:
>> 2016-05-05 20:44 GMT+02:00 Jelmer Vernooij <jelmer at jelmer.uk>:
>> > On Fri, May 06, 2016 at 06:33:45AM +1200, Andrew Bartlett wrote:
>> [...]
>> > However, the work to get to get the test runs stable should really
>> > happen
>> > upstream and not in the Debian packaging. This is not specific to
>> > Debian.
>> > The upstream testsuite shouldn't fail regularly. Until it is stable
>> > there,
>> > I don't see a point in trying to get it to run on Debian.
>>
>> I agree. but currenly the quicktest only fails on the raw.write
>> tests,
>> under 32-bit only.
>>
>> I've removed this test (actually 3 tests) on my local repo, currently
>> rebuilding on i386.
>>
>> If everything goes well, I'll upload this. Let's see if the flappy
>> test is still here (it was on amd64 reproducible buildd).
>>
>> I'm sorry that my uploads have stopped samba 4.4 to go to testing.
>> Will fix this soon.
>
> I'm really sorry, but we need to first work on this upstream, then in
> experimental.  There are still too many flapping tests to do this
> reliably in something like Debian, certainly for unstable.
>
> Have a read of
> https://lists.samba.org/archive/samba-technical/2016-May/113894.html fo
> r the sorry state of affairs.
>
> We first need to get this reliable upstream, then in experimental, as
> Jelmer and I have discussed.  It is a very worthwhile task, but not a
> small one.
>
> Because of the nature of the Samba flapping tests, you need to have a
> large sample, not just if a build works once.
>
> To be clear, I'm currently running 10 builds in large VMs on the
> Catalyst cloud to try and get a reasonable statistical overview of the
> tests for a bug I'm fixing at work.  If the builds run on slow or busy
> hosts, we can expect spurious failures.

THis is sad.

I'll upload again with make quicktest return code ignored (once built locally).

Regards

-- 
Mathieu