[Piuparts-devel] Pending mass bug filing for broken symlinks detected by piuparts

[Andreas Beckmann]

On 2013-06-02 20:26, Dave Steele wrote:
> On Sun, Jun 2, 2013 at 9:54 AM, Andreas Beckmann <anbe at debian.org> wrote:
>> So let's dig into the lists (only the .csv you sent is usable),
> 
> So you retried the spreadsheet after I changed permissions, with no
> success? It sure would be great if you could resolve some of these
> issues directly with less discussion needed.

the google docs spreadsheet still wants a login

>>   Tag: broken-symlink-libdevel-so
>>   Severity: serious
>>   Broken link: /usr/lib/libFOOBAR.so -> libFOOBAR.so.42
>>   Problem: libFOOBAR-dev package with libFOOBAR.so link misses
>>            Depends:libFOOBAR42
>>   Reason: forgotten dependency
>>   Reason: multiple libraries built from src:FOO, maybe split into
>>           separate packages after some time, but the (single) -dev
>>           package was not adjusted to depend on all packages
> 
> So what you've really described here is the case of
> broken-symlink-libdevel-so-forgotten-dependency, to be joined by
> broken-symlink-libdevel-so-expects-recommends and many other brethren,
> right?
> 
> If I can change this 'Reason' in the category definition to say
> something like "forgotten dependency, missing Recommends, or link to
> reverse dependency", pointing to a category broad enough to cover all,
> we may be fine.

I don't think Recommends are a useful relation for -dev packages, for
the 2 packages I looked at this is clearly a missing Depends.

> Otherwise, we may have a flat out disagreement here. The bug filing
> has a responsibility to verify that the problem exists, that it is
> assigned to the proper package, and that the severity is correct. For
> these shared library issues, that can be satisfied by verifying that
> the package created the symlink (I believe this is done), that it
> sources from global shared library directory (need to verify link vs.
> target here), and that the symlink is broken (piuparts does a fine job
> at that). The bug filing does not need to have a requirement to go
> into further analysis of the mechanism of the breakage, or to point to
> the particular related package involved with the problem (if there is
> one), especially if that requires manual research on EACH OF 86
> PACKAGES. If it can be automated without too much trouble, that is
> another question, that I'll look into a bit.

since you know the broken symlink, you know the expected target file and
can look this up in the Contents file

grep "$(python -c 'import os,sys; print
os.path.normpath(os.path.join(os.path.dirname(sys.argv[1]),
sys.argv[2]))[1:]' $link  $target) " /tmp/Contents-sid-main-amd64

that's a one-liner, you can even filter it through
| awk '{ print $2 }' | cut -d/ -f2
to get the binary package name (that last part does not work cleanly if
the file is provided by more than one package, but that should be rare)

Using perl+sed for parsing the .csv and using a sh loop over all broken
links is left as an excercise to the reader.

>> And for filing a serious bug, a manual inspection of the package should
>> be made, and as a result we should be able to mention the missing
>> dependency in this case.
> 
> Disagree. I think we can trust that piuparts identified the right
> package, for cases in sid-fail-broken-symlinks where the broken
> symlink appears only once, and also that it is not a requirement to
> identify the missing dependency/reverse dependency/recommends in the
> bug filing.
> 
> I have gone beyond this, by inspection at the spreadsheet level that
> the symlink ownership by the package 'makes sense'. You've pointed out
> below that a toolchain could be responsible. Spreadsheet-level
> inspection is a reasonable filter for catching these.
> 
> Yes, it is possible for there to be cases in THE 86 PACKAGES that the
> expertise of the maintainer will be involved in targeting the
> ultimately right package. I do not believe that is inappropriate.
> 
> Perhaps we need a mechanism for arbitrating this? Or a plan that
> recognizes that I am not up for doing this for 86 PACKAGES. Otherwise
> we could be looking at another brick wall.
> 
> As a separate issue, this raises additional questions on how you wish
> to list the maintainers section. That remains open.
> 
>> That seems to cover most of the broken links in /usr/lib, and I fully
>> agree to file them as serious.
>> I would be a bit more careful with broken links in "package specific"
>> directories like /usr/lib/PACKAGE/, but if a missing dependency is
>> obvious, that should be serious as well.
>>
> 
> Ok. I'll look harder at that.
> 
>>
>> Looking at xmanpages-ja from your list, that is also serious:
>>
>>  /usr/share/man/ja/man3/XLockDisplay.3.gz ->  XInitThreads.3x
>>
>> the link target is missing the .gz extension (and policy says somewhere
>> that links to compressed files must have the same extension)
>>
> 
> Ok, will research. That would be the second category, in my book.
> 
>> rebuilding in sid fixes this issue
> 
> Guaranteed? Only if dh_installman is doing the work, right? Is this
> because dh_installman changed behavior recently? Otherwise, more
> explanation please.

I tried it. rebuild-in-sid is a handy local script (all it does is
apt-get source && pbuilder build with appropriate options).

>> there are lots of broken links to jquery.js and friends - please
>> investigate, this may be a toolchain bug, too (or a "missed" transition)
>> (this should be analyzed in stable, too)
> 
> I did look at this, for at least a sampling of cases. jquery was a
> recommended package.

maybe we should verify the documentation related ones with
--install-recommends

>> * a bug should be filed for xmanpages-ja, no d-d discussion needed,
>> intent-to-NMU
> 
> Will prepare bug(s). I don't intend to NMU (can't actually - you want
> this?). xmanpages-ja will still come up again at the 'normal'
> conversation.

I'll NMU if needed, not sure if the maintainer is still active.

>> * we should have a specific d-d discussion about filing broken .so
>> symlinks in -dev packages as serious - how many would that be?
> 
> Glad you asked :-). 82 BINARY PACKAGES, 66 source at last count.
> 
> Sorry, you said -dev packages. I can't right now. Try grepping the csv
> on serious and -dev.

there should not be a big difference

>> Rationale: a -dev pkg that cannot be used to link against its library is
>> useless
> 
> or more simply a violation of "Packages containing shared libraries
> must be constructed with a little care to make sure that the shared
> library is always available". A violation is a violation. Serious is
> serious. More nuance is not required.

-dev packages (Section: libdevel) don't contain shared libraries. The
provide only the .so link needed at link time to find the real shared
library ...

>> For the bug template:
>> Subject: $PKG: Broken .so symlink due to missing Depends: $DEP1,$DEP2,..
>>
> 
> Per previous objection, I see this as
> 
> Subject: $PKG: Broken .so symlink

getting the package providing the wanted file is easy :-)

Andreas