[Piuparts-devel] Reporting piuparts results

[Lars Wirzenius]

Holger Levsen is working on getting piuparts.cs.helsinki.fi (a.k.a.
piatti.debian.org) running piuparts regularly again. This brings up the
question of what to do with the results: http://piuparts.cs.helsinki.fi/

The basic issue here is how to get package maintainers pay attention to
results from automatic testing tools.

When I was running piuparts, I would look at each log file from a failed
test and manually report a bug. The main reason for this was that this
seemed to me to be a fairly efficient way of getting the problems
actually fixed, around half of the bugs I filed were fixed. It is,
however, very labor intensive.

Piuparts generates lots of log files. Going through all logs from failed
tests manually requires many hours of work every week. Worse, it is
pretty repetitive work, and yet it is not always easy to automate.

The lintian approach is to assume every developer runs lintian
themselves. There is also a lintian.debian.org site, but it seems mostly
useful for those who want to see a bigger picture. The success rate of
lintian is not bad, but I feel it could be better: it'd be nice if
problems lintian knows about in packages wouldn't persist for years,
which sometimes happens.

What should we do with piuparts reports? Is there a middle ground
between labor-intensive manual bug reporting and wishful thinking that
all developers will start running piuparts?

Would a weekly summary of which packages have a problem, sent to
debian-devel, be a good compromise? I suspect most people would still
just ignore it.

Perhaps the Package Tracking System could be extended with a new
keyword, like "automatic-qa" or such. A weekly automatic mail sent there
from piuparts.cs.helsinki.fi listing the package's QA status. Could
include, say, lintian and linda results, too.

-- 
Never underestimate the power of a small tactical Lisp interpreter.