[Piuparts-devel] Reporting piuparts results

[Lars Wirzenius]

On ti, 2007-01-02 at 15:14 +0100, Lucas Nussbaum wrote:
> The main problem with piuparts is that it finds too many problems, some
> of them more serious than others. I found it easier to hack it so that
> it only finds a smaller set of problems.

It would be good to change piuparts so the user can choose which tests
to do and which not to do.

> This way, one only consider very similar failures, and the number of
> logs to analyze for each piuparts run gets much smaller.

When I developed piuparts, I started with a small set of tests, and ran
it on the archive, and reported any problems I found, and then I
implemented more tests, re-ran everything, etc. It didn't seem to matter
much to me: there was lots of logs to review anyway.

However, unless something better comes up (say, making piuparts reliable
enough to make bug reports automatically), testing things piecemeal is a
reasonable strategy.

> I'm also interested in turning such tasks (logs reviewing) into a more
> collaborative process. I created an alioth project[1] for this. A good
> start could be to maintain inside SVN a list of packages that cannot be
> tested using piuparts. Please join the alioth project if you are
> interested to work on this...

My current thinking is more on the lines that it's too much work for any
small group to be manually reviewing anything, when you're testing the
entire archive. It's labor intensive, and if it can't be automated, it
should be massively distributed. With that I mean that every package
maintainer should be reviewing their own logs. I've no idea how to make
that happen, though.

-- 
Talk is cheap. Whining is actually free.