[Piuparts-devel] Piuparts state-dependency-failed-testing analysis

[Dave Steele]

I find myself frequently called back to the Piuparts waiting-to-be-tested
package list at
http://piuparts.debian.org/sid/state-dependency-failed-testing.html, and got
curious about what is keeping that list as long as it is. Following is an
analysis.

There are currently close to 3000 packages
in state-dependency-failed-testing that can be traced to one or more
state-failed-testing packages. That number actually went down quite a bit in
the last week, when libgtk2.0-bin passed, clearing hundreds of packages for
testing.

There are less than 300 packages in state-failed-testing holding up those
tests. The blocking distribution is typically very skewed. At any given
time, half of the packages in state-dependency-failed-testing may be held
there, directly or indirectly, by 3 to 5 packages. Eliminating libgtk2.0-bin
last week exposed libreadline6 as the new top blocker.

Here's the output of a script that scrapes
http://piuparts.debian.org/sid/state-dependency-failed-testing.html and
analyzes blocking packages:

# ./piublocker
dependency failed -  2899
failed testing -  274

blocking free cum  package
1588    1005  1894 libreadline6
 451      35  1854 sgml-data
 429       1  1466 docbook-xsl
 191     153  1286 texlive-base
 155      79  1182 ca-certificates-java
 125     106  1071 php5-cli
  89      29  1042 gnustep-base-common
  60       0   982 gnustep-back0.20
  49       6   973 menu
  49       4   957 libcommons-httpclient-java
  42       0   915 python-ibus
  40      15   897 libcommons-beanutils-java
  35      34   862 gosa
  35       1   827 blends-common
  33      33   794 liquidsoap
  30       3   791 libspring-core-java
  28      20   771 libwebkit1.1-cil
  28       1   750 libatinject-jsr330-api-java
  26      26   724 drupal6
  26       7   703 antlr
  24      24   679 cyrus-common
  21      13   664 libmx-1.0-2
  20      15   645 openssh-client
  17       0   630 libhttpclient-java
  17      17   613 drizzle
  16      16   597 dovecot-core
  14       0   597 sat4j
  14       6   591 proftpd-basic
  14       0   577 libjasper-java
  14      11   566 libhttp-date-perl
  14       7   559 libgcj-bc
  13       4   546 libmono-addins-gui0.2-cil
  12      10   535 postfix
  12       3   523 libusrp0
  12       9   512 libgtkhtml-4.0-common
  11       6   501 libncurses5-dev
  11       6   490 libgtkglext1
...

This output says that there are currently 2899 packages
in state-dependency-failed-testing traceable to a state-failed-testing
package (that doesn't exactly match Piupart's count of 2920). 274 packages
are responsible for that blocking. More than half of them (1588) are blocked
by a single package, libreadline6. 1005 of those packages would be cleared
for testing by removing only libreadline6 from the list of blockers.
Possibly, at least some of those exposed packages may have blocking numbers
in the 1000 range (e.g. 'python' is in that list).

The "cum" shows how many packages are left as each subsequent top blocker is
cleared. From that number, you can see that the dependency list could be cut
in half by removing the top 4 packages on the list. 80% would be cleared by
passing the top 29 packages.

It seems worth the effort to identify and focus fixing the top packages on
this list, over time. There should be opportunities for low-hanging fruit
whenever the state-dependency-failed-testing list gets over about 1000-1500
entries.

The source for the script piublocker is at
https://github.com/davesteele/piublocker
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/piuparts-devel/attachments/20111022/6e0c8dfe/attachment.html>