[Piuparts-devel] RFH: piuparts tests for lenny

[Lucas Nussbaum]

Hi,

During Debcamp/Debconf, I wanted to work on testing the archive with
piuparts, in preparation for the lenny release. This was done prior to
the etch release, and allowed to find a lot of RC bugs.  Unfortunately,
I didn't have enough time to finish this work, and I won't have time to
work on that before the lenny release. The goal of this mail is to
describe the current status, so someone else can pick up this task.

I had two goals:
(A) improve piuparts, to add a report summarizing the result of the test
    at the end of the log. Currently, piuparts' output is difficult to
    parse, and also difficult to understand for humans, if they are not
    very familiar with what piuparts does.
(B) reduce the list of "suspect" packages to a small amount, so someone
    can analyze the remaining packages without needing any special
    computing power.

(A) was mostly done by Tobias Grimm and Asheesh Laroia, and resulted in
changes which I commited to a branch in the piuparts svn repository:
svn+ssh://svn.debian.org/svn/piuparts/people/lucas/piuparts-report
It's a good basis, but it's not perfect yet. The current status is
summarized in debian/changelog:
    Remains to be done:
    + include output of install/removal/purge test in the report.
    + generate report by default. There's no point in not outputting it.
    + non-fatal failures should be non-fatal by default.
    + output the report to the end of the log, not to a seperate file.
    + general cleanup.
You can use "my" branch to work on that, I don't mind if you commit to
it directly.

(B) resulted in a list of 914 packages that fail an
install/removal/purge piuparts test. I wasn't interested in upgrade
tests: complex to test, and the former needs to be done first anyway.
The list can be found at http://blop.info/pub/piuparts-suspect-packages.txt
The list contains packages that didn't really fail the piuparts test,
but only failed because of a problem in my setup. But all packages not
included in the list are fine.
For etch, the following failures were considered RC, and it probably
makes sense to keep the same criteria for lenny:
piuparts failures:
- not caused by added/removed/modified files, or broken symlinks.
  (so run piuparts with --warn-on-others --no-symlinks)
- not caused by the missing installation of packages that almost
  everybody has installed anyway (ucf, debconf, adduser come to mind).
  (so run piuparts with a custom --debfoster-options string)

As I said, I won't be able to work on that (will be busy with day job).
But if you have questions, just ask.
-- 
| Lucas Nussbaum
| lucas at lucas-nussbaum.net   http://www.lucas-nussbaum.net/ |
| jabber: lucas at nussbaum.fr             GPG: 1024D/023B3F4F |