[Piuparts-devel] testing sid with --warn-on-leftovers-after-purge, too (was: Re: pull: fixes)

[Andreas Beckmann]

On 2012-12-23 18:40, Holger Levsen wrote:
> On Freitag, 21. Dezember 2012, Andreas Beckmann wrote:
>> PS: what about running two sid tests on piatty: with and without
>> --warn-on-leftovers-after-purge?
> 
> hm, do we really need to test almost 40k extra packages just to get this 
> difference?!

The good thing: seeding is easy, we can copy (better hardlink) the pass
logs from current sid.

stats sid --warn-on-leftovers-after-purge:
	failed		99
	dep-failed	41
	dep-cannot	25

stats sid classic (fail-olap):
	failed		180
	dep-failed	287
	dep-cannot	23
	(waiting	14, so above numbers could be a bit higher)


test coverage with --warn-on-leftovers-after-purge is better (by 350
packages) which should better help against serious errors in untested
packages migrating to testing

test coverage on piatti will be a little lower since I'm running with
experimental --enable-database

In my setup I currently have
	sid
	sid-lo  (with leftover checks, äquivalent to piatti's sid)
	sid-iri (--install-remove-install, 3 more failures than sid)
	sid-ipi (--install-purge-install, many false positives)

and for piatti I would propose
	* rename sid to sid-lo (or any other name)
		--warn-on-leftover-after-purge --install-remove-install
		* delete the additional iri fail logs
	* add a new sid without these two options
		* seed it with the pass logs from sid-lo
modify the PTS to report two different failures on sid (and skip the
sid-lo failure if there is a sid failure)

Andreas