[Piuparts-devel] Bug#674498: Display reverse dependency metrics

[Dave Steele]

Package: piuparts
Version: 0.44
Tags: patch

piuparts.d.o will not test a package if one or more of its
dependencies have previously failed, or is in turn blocked by a
failure. Failed packages with high reverse dependency counts can
therefore cause a large number of packages to be blocked from testing.
Better visibility of reverse dependency counts, and of blocking
metrics, could lead to improved test coverage.

The git 'rrdeps' branch below adds optional recursive reverse
dependency count information to packagesdb.py, and implements a
demonstration concept for displaying the information in the
piuparts-report.py output. It also calculates and stores the count of
packages blocked, for packages which are in a failed state. There are
(currently) 4 unique commits on this branch.

    https://github.com/davesteele/piuparts/commits/rrdeps

The output is two integer count fields on each of the Package objects.
The calculation adds about 30 seconds to the piuparts-report setup of
the PackagesDB object against the sid Packages file. Memory needs
during the calculation are modest. 'piuparts-master' performance is
not affected.

The piuparts-report changes modify the failed-testing and
cannot-be-tested summary pages to display reverse dep count and block
count for each 'bad' package, and to sort by block count. None of the
other pages are affected.

I've tested this against the sid section.

Here's a text dump of a section of the piuparts-report html output,
with this change:

    Packages in state "failed-testing" in sid (reverse deps, blocked pkgs)

    * docbook-xml (478,474) (Debian XML/SGML Group
<debian-xml-sgml-pkgs at lists.alioth.debian.org>)
      o dependency sgml-data is successfully-tested
      o dependency sgml-base is successfully-tested
      o dependency xml-core is successfully-tested
    * libvlc5 (494,454) (Debian Multimedia Maintainers
<pkg-multimedia-maintainers at lists.alioth.debian.org>)
      o dependency libc6 is essential-required
      o dependency libvlccore5 is successfully-tested
    * libsdl1.2debian (832,357) (Debian SDL packages maintainers
<pkg-sdl-maintainers at lists.alioth.debian.org>)
      o dependency libasound2 is dependency-failed-testing
      o provider liboss4-salsa-asound2 is dependency-failed-testing
      o dependency libc6 is essential-required
      o dependency libcaca0 is successfully-tested
      o dependency libdirectfb-1.2-9 is dependency-failed-testing
      o dependency libpulse0 is dependency-failed-testing
      o dependency libx11-6 is dependency-failed-testing
      o dependency libxext6 is successfully-tested
      o dependency multiarch-support is failed-testing
  ...

Disregard the actual packages listed - I'm getting spurious errors
from using a caching proxy.

There have been some heavy hitters passing through failed-testing in
the last few weeks. This makes them much more visible.

The changes are branched off of piuparts-server (which BTW has a small
update). I can move them if that is helpful.