[Piuparts-devel] piuparts results for use in britney

[Niels Thykier]

Andreas Beckmann:
> [ moving this part of the discussion to piuparts-devel@ with Niels
> Cc:ed, since I don't know whether he is subscribed ]
> 

Thanks, please keep me CC'ed. :)

> On 2016-12-19 17:33, Holger Levsen wrote:
>> And "anyway", considering piuparts results in sid are now considered for
>> testing migrations, shouldnt we test sid+stretch with the same flags?
> 
> That might be neccessary ...
> 
> Niels, is a "piuparts regression" a "pass" in stretch and a "fail" in
> sid? Do you take testing2sid into account?
> 

A regression is defined as a fail in sid and a "non-fail" (e.g. "X" or
"W") in testing.  We currently only look at the sid and testing suites
respectively.

Though it should be possible to also review the testing2sid, if we can
properly define how britney should merge it into the results.  The
current code is at:


https://anonscm.debian.org/git/mirror/britney2.git/tree/britney2/policies/policy.py#n543

With tests at:

https://anonscm.debian.org/git/mirror/britney2.git/tree/tests/test_policy.py#n169

(Input data is the "piuparts.d.o/<suite>/summary.json" files)

> Would further upgrade test suites be helpful? I'm thinking about
> jessie2stretch and stable2sid (the latter may come with a lot of false
> positive failures depending on the "stability" of sid, but sometimes it
> catches some bits that work in testing, sid, testing2sid, but will break
> in stable2testing only *after* migration; so far I run it on my local
> piuparts instance only)
> 

I am interested in this, but probably first for buster.  There is
already some discussion on undoing the current setup for stretch.

Regardless of what suites we track, we will be interested in the
property that if a package with a regression migrates (due to a hint),
then the next version should never be a regression.  After all, that is
what sparked in this thread. :)

>> (and maybe introduce a sid-strict suite?)
> 
> I like that name :-) and might follow up with some patches ... and might
> add some flags.
> 
> 
> Andreas
> 

Sounds good to me :)

Thanks,
~Niels