[Pkg-electronics-devel] preparation of gnucap-python 0.0.2-2 (Re: Processed: python3.6)

[Felix Salfelder]

Dear Carsten.

thanks for your reply.

On Thu, Jan 17, 2019 at 07:01:45PM +0100, Carsten Schoenert wrote:
> > I used "dh_shlibdeps -- -v" to see all of them.
> 
> This will not help here, it make the situation even "worse" as you will
> get flooded by more lines about the basically same issues.

i merely used the command above to see all the warning, and in order to be able
to skim through. i did not put that into rules.

> I wouldn't call this correct behavior, at least on my side some bells
> are ringing as for me such an output isn't normal and show that
> somethings isn't working as it should. For me this is using the tools
> not correctly.

I beg to differ. There is a heuristic involved and the heuristic isn't
perfect. as it stands, the manual tells me to ignore the warnings, under
some conditions. perhaps the explanation of the example below will
clarify the issue a bit more.

> What about compiler warnings? These are also just warnings. Do you
> ignore them also, obviously not. :)

i don't think this is a good comparison. the case has more in common
with lintian warnings. these can be overriden in case they are wrong,
which is useful, if they are known to be of heuristic nature.

> > afaict, there is no package dependency implied by these symbols.
> 
> Of course not, simply because dh_shlibdeps can't extract which these
> would be. And that's the danger about I've written. You need to prove
> that the output is correct.

then s/implied/intended/. dh_shlibdeps can't extract and is not meant to
extract dependencies from those unresolved symbols.

what do you mean by "prove that the output is correct"? is there any
doubt?

> > good to know. However, -l will only help resolving local symbols, such
> > as in private libraries, which do not seem to be the issue.
> 
> You are sure?

The first thing about -l is what the manual says. the second i checked: none of
the symbols of libgnucap-python.so show up in the warnings. all dependencies to
the package containing libgnucap-python.so are correctly created.

> > dpkg-shlibdeps: warning: debian/python3-gnucap/usr/lib/python3/dist-packages/gnucap/_component.cpython-37m-x86_64-linux-gnu.so contains an unresolvable reference to symbol _ZN9COMPONENT9tr_reviewEv: it's probably a plugin

_component.cpython-37m-x86_64-linux-gnu.so is a plugin, which is meant
to be used in the presence of libgnucap symbols.

$ readelf -s /usr/lib/x86_64-linux-gnu/libgnucap.so.0 \
        | grep _ZN9COMPONENT9tr_reviewEv | c++filt
1108: 000000000009bac0   141 FUNC    GLOBAL DEFAULT   12 COMPONENT::tr_review()

this is why libgnucap is dlopen'ed with RTLD_GLOBAL just before the
plugin is imported. see gnucap/__init__.py. FWIW, there is no provision in
python for bypassing __init__.py.

if you dlopen the plugin from some program without this symbol
(intentionally or not) you may well get an error. that depends on what
dlopen flags are set and which symbols you are trying to access in the
plugin. Package dependencies are not going to stop you or save you.
hence, those unresolved symbols in plugins have to be ignored when
calculating the dependencies.

> > $ git grep "tr_review()"
> > gnucap/_e_cardlist.i:   TIME_PAIR  tr_review();
> > gnucap/_e_elemnt.i:  TIME_PAIR tr_review();
> > gnucap/s_tr_swp.cc:  TIME_PAIR time_by = CARD_LIST::card_list.tr_review();
> > misc/d_coil.cc:  TIME_PAIR tr_review()          {return TIME_PAIR(NEVER,NEVER);}
> 
> So this symbols comes from your code I guess.

the tr_reviews you found are prototypes (.i is some kind of header) and
a call (in s_tr_swp.cc) to a function in libgnucap and a virtual
override (in d_coil.cc). It does not come from my code. the unresolved
symbol reported above and overridden in d_coil.cc is implemented in
libgnucap (e_compon.cc). CARD_LIST::tr_review() is also implemented in
libgnucap, (e_cardlist.cc)

> I've no idea where this ends and in which binary. But that part is
> missing for dh_shlibdeps.

not missing really. there is an unresolved symbol which is meant to be
unresolved and should be simply ignored. shlibdeps says the file is
possibly a plugin (yes it is!) and the manual says that we should ignore
the warning, after checking whether it actually is a plugin (see end of
this mail).

I could filter out the warnings with some grep -v, but I am not
convinced about the benefit. doing so will not solve the actual problem.
A better heuristic should suppress these warnings in the first place.

NB: One more thing the manual suggests is to drop the SONAME from the plugin,
in the hope that some other heuristic kicks in. while the SONAME is
unnecessary, it is certainly not wrong or harmful by itself, or the source of
our problem. Either way, this has to be done upstream, and I will try before
0.0.3.

> > [the hack]
>
> For now it's o.k. to live the current output, but it should really get
> some attention in the future.

that's good.

regards
felix

==== from dpkg-shlibdeps(1), WARNINGS

 binary contains an unresolvable reference to symbol sym: it's probably a plugin
		  The  indicated  symbol has not been found in the libraries linked with the binary. The
		  binary is most likely a plugin and the symbol is probably provided by the program that
		  loads  this  plugin.  In  theory a plugin doesn't have any SONAME but this binary does
		  have one and as such it could not be clearly identified as such. However the fact that
		  the  binary is stored in a non-public directory is a strong indication that's it's not
		  a normal shared library. If the  binary  is  really  a  plugin,  then  disregard  this
		  warning. But there's always the possibility that it's a real library and that programs
		  linking to it are using an RPATH so that the dynamic loader finds it.  In  that  case,
		  the library is broken and needs to be fixed.