[Piuparts-devel] using sid-strict etc for britney

[Ivo De Decker]

Hi,

On 1/19/19 5:41 PM, Niels Thykier wrote:
> Holger Levsen:
>> Hi,
>>
>> Andreas sent this to the piuparts-devel list while I think it's more
>> appropriate to discuss this on debian-release at ....
>>
> 
> Hi Holger,
> 
> Thanks for forwarding this idea. :)
> 
>> On Thu, Jan 10, 2019 at 02:38:38PM +0100, Andreas Beckmann wrote:
>>> currently britney uses piuparts regressions comparing testing vs. sid to
>>> block migrations. That's fine, since we should have few false positives
>>> in sid nowadays.
>>> It would be nice if we could use a similar scheme like autopkgtests uses
>>> to delay migration for other tests, like sid-strict or testing2sid.
>>> As in "if your package leaves files after purge, you don't get the
>>> migration reduction from 5 to 2 days"
>>> and for testing2sid (or stable2sid) there may be more false positives
>>> caused by other packages, so it would be great to at least delay
>>> migration by a few days as well, to allow manual inspection of the
>>> failure, I several times didn't manage to file a bug within 2 days of
>>> upload to prevent migration of a buggy package to testing ...
>>
>> sounds like a splendid idea to me!
>>
>> How can we get this going?
>>
>>
> 
> At its essence, it is a question of updating the PiupartsPolicy in
> Britney[1] along with adding relevant test cases (either in "tests/" or
> in britney2-tests[2]).
> 
> Once that is done, we can merge the code and update the britney wrapper
> to fetch the relevant summary.json files from piuparts.d.o.
> 
> Hints as to the actual steps:
> 
>   * PiupartsPolicy.initialise should load the relevant summary files.
>     (Difficulty level: copy-paste + rename things)
> 
>   * PiupartsPolicy.apply_src_policy_impl should analyse the new files
>     and determine which delay to apply if any.  If a age delay is to be
>     applied then the method should now call:
>       excuse.add_penalty('piuparts', <delay-in-days-goes-here>)
> 
>   * TestPiupartsPolicy should be updated to include a case for it.  It
>     might make sense to have PiupartsPolicy include a note in the
>     policy_info so the test case can rely on that (rather than having
>     to setup an AgePolicy as well for verification).
> 
> We have a ".gitlab-ci" on the git repo that runs all the basic test
> cases for you on push to salsa.  Alternatively, have a look at the
> ci/gitlab-ci-runner for how to run the tests manually.

On a related note:

- It would be nice to have detailed information about the binaries 
tested in the export, this would allow britney to use this information 
for binNMUs. Piuparts already tests binNMUs, but the info in the export 
only lists the source, not the binary (or the binary version), so that 
info isn't available yet. This would probably mean an incompatible 
change to the export (which can exist side by side with the current 
one). Do you have an idea about what services are processing this file? 
I know at least britney, tracker, udd and the old PTS have info from 
piuparts.

If the export is changed anyway, it could contain more detailed 
information about what tests failed and the severity of the failures. 
This might allow an easier implementation of the diversified result 
(block, delay, ...)

- Currently, it looks like buster-proposed-updates isn't tested by 
piuparts (which isn't a big deal, as there are no packages in there 
yet). Could this be added as a suite (together with buster2proposed)?

Thanks!

Ivo