[Piuparts-devel] Bug#698526: Sort known issues by reverse dependency count

[Dave Steele]

Package: piuparts
Severity: wishlist
Tags: patch
thanks

Packages with high reverse dependency counts can cause "known problem"
issue lists to balloon. Providing rdep visibility in the issue report
can highlight these problems, making it much easier to pare the list
down.

The "well-known" git branch implements a version of
detect_well_known_errors to accomplish this. The script is ported from
bash to python, to take advantage of the rdep capability of
piupartsdb. It was developed alongside the bash script to support
side-by-side testing.

https://github.com/davesteele/piuparts/commits/well-known

427aa41 detect_well_known_errors - restore recheck and recheck-failed options.
a4553bc Bump the required python version to 2.7.
a12f676 detect_well_known_errors - display the reverse dependency count.
500e97f detect_well_known_errors - sort known errors/issues by rdep count.
7de4eb9 detect_well_known_errors - integrate the package templates.
d066bb3 detect_well_known_errors - Template HTML format fix.
76b8ce2 detect_well_known_errors - Copyright notice.
b8af3e4 detect_well_known_errors.py - move to detect_well_known_errors.
25b9351 Remove bash detect_well_known_errors.
ece5e4e detect_well_known_errors.py - change ext's to create kpr and tpl files.
39837e9 detect_well_known_errors.py - print failures to match bash script
198c65e detect_well_known_errors - Create html tpl files
4049338 detect_well_known_errors.py - Create Failure Mgr class to hold kpr fails
d04c1bb detect_well_known_errors.py - Create missing kpr files
1880598 detect_well_known_errors.py - add class for handling known problems
9b25943 detect_well_known_errors.py - establish the problem file location.
53df049 detect_well_known_errors.py - clean obsolete kpr and bug files
cdd8803 Add skeleton for python replacement of detect-well-known-errors
601e6a7 start with 0.50
df94975 release as 0.49


Here's a partial text dump of the resulting broken_symlinks issue html
output (note the extra rdep parameter for each package):

=======================

Packages which have logs with the string "Broken symlinks" in sid,
sorted by reverse dependency count.

This is clearly an error, but as there are too many of this kind,
piuparts can be configured to not fail if it detects broken symlinks.
Another option is not to test for broken symlinks. See the piuparts
manpage for details.

The commandline to find these logs is:

COMMAND='grep -E "(WARN|FAIL): Broken symlink"'

Please file bugs!

pass/python-support_1.0.15.log (1701) (BTS)
pass/dictionaries-common_1.12.10.log (1343) (BTS)
pass/aspell_0.60.7~20110707-1.log (1107) (BTS)
pass/aspell-en_7.1-0-1.log (1043) (BTS)
pass/libenchant1c2a_1.6.0-7.log (1042) (BTS)
pass/python-numpy_1:1.6.2-1.log (765) (BTS)
pass/vlc-data_2.0.5-1.log (615) (BTS)
pass/libvlccore5_2.0.5-1.log (614) (BTS)
pass/libvlc5_2.0.5-1.log (612) (BTS)
pass/vlc-nox_2.0.5-1.log (604) (BTS)
pass/phonon-backend-vlc_0.6.0-1.log (585) (BTS)
pass/phonon_4:4.6.0.0-2.log (583) (BTS)
pass/python-cairo_1.8.8-1+b2.log (562) (BTS)
pass/python-gtk2_2.24.0-3.log (509) (BTS)
pass/kate-data_4:4.8.4-1.log (486) (BTS)
pass/katepart_4:4.8.4-1.log (485) (BTS)
pass/kde-runtime-data_4:4.8.4-2.log (484) (BTS)
pass/kdelibs5-plugins_4:4.8.4-4.log (484) (BTS)
pass/kde-runtime_4:4.8.4-2.log (483) (BTS)
pass/libgs9-common_9.05~dfsg-6.3.log (336) (BTS)
pass/libgs9_9.05~dfsg-6.3.log (335) (BTS)
pass/tex-common_3.15.log (292) (BTS)
pass/texlive-common_2012.20120611-5.log (234) (BTS)
pass/python-simplejson_2.6.2-1.log (219) (BTS)
pass/texlive-doc-base_2012.20120611-1.log (216) (BTS)
pass/texlive-binaries_2012.20120628-4.log (198) (BTS)
pass/luatex_0.70.1.20120524-3.log (193) (BTS)
pass/texlive-base_2012.20120611-5.log (192) (BTS)
pass/libkresources4_4:4.8.4-2.log (183) (BTS)
pass/libkabc4_4:4.8.4-2.log (182) (BTS)
pass/libkcal4_4:4.8.4-2.log (180) (BTS)

========================

As a side effect, .tpl file creation is about 30X faster, down from
about 1 1/2 minutes. Much of that improvement is used up by the rdep
sort calculation in piupartsdb. .kpr creation is still very slow. Some
serious additional speed improvements can be achieved by 1) replacing
the subprocess shell calls for grep with re, and 2) incorporating
detect_well_known_errors into piuparts-report, allowing a piupartsdb
object to be shared between the two.