[Pkg-electronics-devel] gnucap package.

[Felix Salfelder]

Hi Carsten.

thanks for your detailed feedback.

On Fri, Mar 23, 2018 at 09:53:52PM +0100, Carsten Schoenert wrote:
> > (i think singular does something very similar)
> 
> No, the intention of this -dev-common package is a bit different to what
> you want to solve.

yikes. arch dependent vs arch independent headers. i haven't considered
any of that.

let me try and address the other things.

> You are here talking about installing possible multiple ABI versions of
> a shared library. Yes, at one point you need to think about such
> constellations, but not right now for gnucap.

good. in a 'normal' situation, the choice of the soname (here:
libgnucap0.so) would prevent future breakage in the case of an upgrade.

i was assuming it would be a good idea to prevent such breakage. i
wouldn't want anybody to link against libgnucap if there is no means of
ensuring that the linked program will keep working for a while
(ideally, indefinitely).

> Of course this is possible, but the upstream source need to provide a
> installation of the files and folders in a way it's usable and doesn't
> conflict with previous project versions. That mostly means you also will
> need versioned symbols within the library (gnucap doesn't do this right
> now).

yes. gnucap also does not add a 0 into the soname. i was assuming that
this would be my job (as a packager). upstream does not make any attempt
to prevent misuse on this level, that includes the choice of the install
paths or config file names.

> A simple example.
> Given you have a package 'mybin' that is installing a binary 'mybin' and
> also libary+header files with an internal version ABI version 0.
> 
> /usr/bin/mybin
> /usr/include/mybin/*.h
> /usr/lib/mybin.so.0.0.0
> /usr/lib/pkg-config/mybin.pc
> 
> For me later as a developer who wants to use the dev files I'm happy to
> find a pkg-config file which holds all information I need, like
> depending other libraries, path to the header files and so on.

in my current approach, both the pkglibdir (here:
/usr/lib/triplet/gnucap0) and the soname of the library (libgnucap.so.0)
are both hardcoded into /usr/bin/gnucap. so if
/usr/lib/triplet/gnucap0/something.so uses symbols from libgnucap.so.0,
that will probably work.

the functionality of pkg-config is in /usr/bin/gnucap-conf (just without
the dependency on pkg-config). i put the config file to /etc, which
turned out to be wrong, as it would break (at least) multiarch, which
pkg-config files do not break.

> Now you create a new much better version of 'mybin' with a ABI change
> but doesn't take care about the correct versioning for the library. With
> which name will the library installed and were? Where will the header
> files go to, which have of course different prototypes for the functions?

the library will be libgnucap.so.1 and the new executable will hardwire
the default pkglibdir to /usr/lib/triplet/gnucap1. plugins there will
use symbols from libgnucap.so.1.

installing the new, "1", version will then not break any binaries for
"0". it will however break plugin compilation (read: loading), unless
the (involved) headers are somehow compatible. if not, i intend to get
away with a (coinstallable) set of headers, similar to your suggestion.

> Yeah, that is a point every upstream project will come to at one day and
> if they do it right this is all no problem. But I don't see gnucap here
> in the next time.

my idea is that /usr/lib/triplet/gnucap0 will avoid most trouble.

> So you don't need to really take care for something
> that isn't a problem currently And even if gnucap comes one day as
> gnucap1, it's not problem to co-install more than one version of the dev
> files.

if the headers end up in libgnucap-dev (which i am arguing against),
then it will be totallt impossible to co-install them. this will become
necessary once somebody has linked against libgnucap.

> /usr/include/mybin1/*.h
> /usr/lib/mybin.so.0.1.0
> /usr/lib/pkg-config/mybin1.pc

> According to the debhelper sequence you currently install the gnucap
> header files and a c++ source file. What is the model compiler?

gnucap-modelgen. turns .model files into .cc and .h. i'm not sure if the
current package installs it, or what the manual says. the .cc and .h
files are obviously useless without the gnucap headers in place.

> > the development files are not (only) associated to a shared library.
> 
> Then they are no development files. The intention of -dev packages is to
> provide all the development files which are needed to create own
> binaries and/or shared libraries.

the headers are needed/used to compile binaries ("plugins"). but these
binaries are not associated/linked to a shared library.

binary compilation happens in the course of "using gnucap" not just
during "development".

> > gnucap will not work without the header files. it will not be able to
> > load plugins, and all (read: most) functionality is in plugins.
> > 
> > that's like python without "import". except python interprets files on
> > its own.
> 
> Sorry I can't follow that. I've tried to find some information related
> to your statement on the current wiki [1] for gnucap. It explaining the
> plugin model that is used, the mechanism that is used for that is
> exactly what I've written about in my previous email.
>
> The binary is loading a shared object file (simply because your work is
> based on the STL and own written functions which are build with a C++
> compiler), but no plain text files (like a header file). I also haven't
> seen such a function in the source. So maybe I oversee something totally
> or we are talking from completely different things.

yes, there's a subletly here. the binary is loading a shared object
(dlopen). that is done by a command, the "attach" command. one possible
attach command is shipped as part of libgnucap.

however, the attach command (like all commands) is a plugin, so it can
replace it (at run time). here's how it works: you load a plugin,
"attach myownattach.so". that provides a different "attach" command, and
will replace the current. then, any subsequent "attach" invocation will
now run whatever you want to.

gnucap-make provides a different attach command. install it (I intend
to package...), run

$ gnucap -a c_make_attach.so    # <= -a is short for "attach".
gnucap> attach myotherpluin.cc  # <= here you go, now loading "plain text"
                                     (not without the matching headers)

> But we don't need to
> dig here into the internal details of loading plugins or other things.
> With a technical view from the Debian side the header files are "just" a
> depending package.
> 
> If your binary need such a header file for what ever reason, but this
> header file isn't needed as a development file, then it doesn't belong
> to the -dev package but to the binary (or a extra) package.

if gnucap-dev does not qualify as a proper -dev package, i'd be happy to
call it gnucap-common.

> What does now the singular package?

perhaps not quite what i thought it does. scrap it (and sorry for the
noise).

> > loading the binary plugins will not need header files. loading the
> > *source* version will. the source version could be in $HOME/my_project/
> > .. a modified version of an upstream plugin, or anything really.
> 
> What is a source version?

the source version would be the plain bare bone .cc file. see above for
how to load it.

> I guess you mean a simulation model? If so we are talking again about
> development files. If a user is building own plugins they are built as a
> shared object file.

no, this is not "development". the compilation step is just inevitable
in normal "use". the intermediate object code does exist but it should
not have any meaning beyond that.

let me pull out that python analogy once more. if i do

$ touch foo.py
$ python -c import foo

that will drop a foo.pyc containing bytecode into my cwd. i did not do
anything related to development for this to happen.

(currently) gnucap calls an external compiler. that is an implementation
detail. the user does not care or is meant to see it. (s)he merely wants
to do somehing like

$ gnucap -a mystuff.so -a mymodel.va
gnucap> ...

(or, for that matter, simply mention mymodel.va in a schematic file)

and just use it.

> I don't hang on the non usage of this prefix '0', if you want to use it
> related to the '0' on the libgnucap0 package I'm fine with this.

i'm a bit worried that nothing in gnucap will hold the user back from
loading binary incompatible plugins. which may lead to random results.
with that extra "0", it will be a bit harder to do that accidentally.
it looks like just a safety net. but with libgnucap-default-plugins1 i
will need the version there anyway.

(note that this is posix, it looks a bit different on non-posix
platforms.)

> > the headers are required to use gnucap.
> 
> strace is telling me something different, but any way. ;)

that's okay. strace does not know that most functionality is in plugins
you did not load. i hope the example above illustrates this.

> gnucap seems to lack a recommends or at least a suggest on the plugins.

yes. thanks.

> > gnucap
> > gnucap0-dev
> > libgnucap0
> > libgnucap-dev (depends on gnucap0-dev, no need for a "0" here)
> 
> Why no '0' then here?

when i link (or package) another gnucap executable, why would i not want
to use (or depend on) the newest library version? libgnucap-dev reflects
my understanding of how it is usually done.

to summarise:
- using gnucap requires headers that must (sufficiently) match the api
  of the library the executable is linked to. ideally they never change,
  but that might be wishful thinking. current work in progress changes
  (yet irrelevant) subtleties. who knows?
- developing gnucap programs need linking with -lgnucap. these programs
  should/must not break upon a library api/abi upgrade.
  arguably "recompile/relink your programs" seems to be very common
  outside debian. i won't recommend to build a production environment on
  this.

I will try and figure it out (hopefully).

cheers
felix