In the past week, the number of state-dependency-failed-testing has increased by 510, to 3409. Of those, about 80% are blocked by the first package in the list, and 75% are blocked only by that package.<br><br><div class="gmail_quote">
On Sat, Oct 22, 2011 at 11:03 PM, Dave Steele <span dir="ltr"><<a href="mailto:dsteele@gmail.com">dsteele@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div><br></div><div>I find myself frequently called back to the Piuparts waiting-to-be-tested package list at <a href="http://piuparts.debian.org/sid/state-dependency-failed-testing.html" target="_blank">http://piuparts.debian.org/sid/state-dependency-failed-testing.html</a>, and got curious about what is keeping that list as long as it is. Following is an analysis.</div>
<div><br></div>There are currently close to 3000 packages in state-dependency-failed-testing that can be traced to one or more state-failed-testing packages. That number actually went down quite a bit in the last week, when libgtk2.0-bin passed, clearing hundreds of packages for testing. <div>
<br></div><div>There are less than 300 packages in state-failed-testing holding up those tests. The blocking distribution is typically very skewed. At any given time, half of the packages in state-dependency-failed-testing may be held there, directly or indirectly, by 3 to 5 packages. Eliminating libgtk2.0-bin last week exposed libreadline6 as the new top blocker.</div>
<div><br></div><div>Here's the output of a script that scrapes <a href="http://piuparts.debian.org/sid/state-dependency-failed-testing.html" target="_blank">http://piuparts.debian.org/sid/state-dependency-failed-testing.html</a> and analyzes blocking packages:</div>
<div><br></div><div><div><font face="'courier new', monospace"># ./piublocker </font></div><div><span style="font-family:'courier new', monospace">dependency failed - 2899</span></div>
<div><font face="'courier new', monospace">failed testing - 274</font></div><div><font face="'courier new', monospace"><br></font></div><div><font face="'courier new', monospace">blocking free cum package</font></div>
<div><font face="'courier new', monospace">1588 1005 1894 libreadline6</font></div><div><font face="'courier new', monospace"> 451 35 1854 sgml-data</font></div>
<div><font face="'courier new', monospace"> 429 1 1466 docbook-xsl</font></div><div><font face="'courier new', monospace"> 191 153 1286 texlive-base</font></div>
<div><font face="'courier new', monospace"> 155 79 1182 ca-certificates-java</font></div><div><font face="'courier new', monospace"> 125 106 1071 php5-cli</font></div>
<div><font face="'courier new', monospace"> 89 29 1042 gnustep-base-common</font></div><div><font face="'courier new', monospace"> 60 0 982 gnustep-back0.20</font></div>
<div><font face="'courier new', monospace"> 49 6 973 menu</font></div><div><font face="'courier new', monospace"> 49 4 957 libcommons-httpclient-java</font></div>
<div><font face="'courier new', monospace"> 42 0 915 python-ibus</font></div><div><font face="'courier new', monospace"> 40 15 897 libcommons-beanutils-java</font></div>
<div><font face="'courier new', monospace"> 35 34 862 gosa</font></div><div><font face="'courier new', monospace"> 35 1 827 blends-common</font></div>
<div><font face="'courier new', monospace"> 33 33 794 liquidsoap</font></div><div><font face="'courier new', monospace"> 30 3 791 libspring-core-java</font></div>
<div><font face="'courier new', monospace"> 28 20 771 libwebkit1.1-cil</font></div><div><font face="'courier new', monospace"> 28 1 750 libatinject-jsr330-api-java</font></div>
<div><font face="'courier new', monospace"> 26 26 724 drupal6</font></div><div><font face="'courier new', monospace"> 26 7 703 antlr</font></div>
<div><font face="'courier new', monospace"> 24 24 679 cyrus-common</font></div><div><font face="'courier new', monospace"> 21 13 664 libmx-1.0-2</font></div>
<div><font face="'courier new', monospace"> 20 15 645 openssh-client</font></div><div><font face="'courier new', monospace"> 17 0 630 libhttpclient-java</font></div>
<div><font face="'courier new', monospace"> 17 17 613 drizzle</font></div><div><font face="'courier new', monospace"> 16 16 597 dovecot-core</font></div>
<div><font face="'courier new', monospace"> 14 0 597 sat4j</font></div><div><font face="'courier new', monospace"> 14 6 591 proftpd-basic</font></div>
<div><font face="'courier new', monospace"> 14 0 577 libjasper-java</font></div><div><font face="'courier new', monospace"> 14 11 566 libhttp-date-perl</font></div>
<div><font face="'courier new', monospace"> 14 7 559 libgcj-bc</font></div><div><font face="'courier new', monospace"> 13 4 546 libmono-addins-gui0.2-cil</font></div>
<div><font face="'courier new', monospace"> 12 10 535 postfix</font></div><div><font face="'courier new', monospace"> 12 3 523 libusrp0</font></div>
<div><font face="'courier new', monospace"> 12 9 512 libgtkhtml-4.0-common</font></div><div><font face="'courier new', monospace"> 11 6 501 libncurses5-dev</font></div>
<div><font face="'courier new', monospace"> 11 6 490 libgtkglext1</font></div><div><span style="font-family:'courier new', monospace">...</span></div>
</div><div>
<font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">This output says that there are currently 2899 packages in state-dependency-failed-testing traceable to a state-failed-testing package (that doesn't exactly match Piupart's count of 2920). 274 packages are responsible for that blocking. More than half of them (1588) are blocked by a single package, libreadline6. 1005 of those packages would be cleared for testing by removing only libreadline6 from the list of blockers. Possibly, at least some of those exposed packages may have blocking numbers in the 1000 range (e.g. 'python' is in that list).</font></div>
<div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">The "cum" shows how many packages are left as each subsequent top blocker is cleared. From that number, </font><span style="font-family:arial, helvetica, sans-serif">you can see that the dependency list could be cut in half by removing the top 4 packages on the list. 80% would be cleared by passing the top 29 packages.</span></div>
<div><span style="font-family:arial, helvetica, sans-serif"><br></span></div><div><span style="font-family:arial, helvetica, sans-serif">It seems worth the effort to identify and focus fixing the top packages on this list, over time. There should be opportunities for low-hanging fruit whenever the </span><span style="font-family:arial, helvetica, sans-serif">state-dependency-failed-</span><span style="font-family:arial, helvetica, sans-serif">testing list gets over about 1000-1500 entries.</span></div>
<div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">The source for the script piublocker is at </font><a href="https://github.com/davesteele/piublocker" target="_blank">https://github.com/davesteele/piublocker</a></div>
</blockquote></div><br>