[RFC] pkg-kde-tools: auto-injections of dependencies

[Pino Toscano]

In data lunedì 7 dicembre 2020 09:01:23 CET, Norbert Preining ha scritto:
> > be at the same version), and that in the past the "big hammer" of
> > long list of breaks in library and -dev packages was attempted. This
> > solution, however, created more issues than it actually solved, and
> > I see the same big hammer solution here creating more hassles than
> > actually solving this.
> 
> I have used the "big hammer" for the TeX Live packages, but there are
> hardly any shared libraries, so obviously my experience is not
> sufficient to properly evaluate the possible problems.
> 
> Could you tell me what are the issues that would we create with the big
> hammer?

As I mentioned:
- too broad solution
- lots of boilerplate in many packages (30+), even in case where there
  is no need for it
- it does not solve the issue with out-of-Plasma packages using Plasma
  libraries

I understand that worked fine for TeX Live, I did not know that.
However, the use cases between TeX and Plasma are different (I'd say
even very much), so I do not think a direct copy&apply without
understanding the problems involved is a good solution.

That's why I asked already: other than those I mentioned, what where
the actual issues that happened? Knowing them is important, as we can
then discuss based on precise issues, and determine whether there might
be more fitting solutions for them. Again, this is not finger pointing,
but rather asking for more knowledge to be able to implement more
effective solutions.

Especially that it adds maintenance costs on the packages, adding more
and more bits even to simple packages. While this seems a minor concern,
these "small bits" need to maintained over time, with newer versions of
dpkg/debhelper/policy/etc. That's why I'd prefer to add the minimum
needed to make sure things work with as less as drawbacks as possible.

> Furthermore, I think this is independent from the very valid issue about
> shared libs and symbol tracking.

Yes and no: as I wrote, what I observed so far was that the majority
(if not all) the issues related to dependencies were related to shared
libraries, in the two categories I wrote in my previous email.

> But let us see what needs to be done **now** if we don't use the hammer ;-)
> 
> Do you think that an upload of the current state of experimental, with
> recent changes you have done to master merged, does make sense? The idea
> would be to make upload to experimental, see how upgrade tests work, and
> make adjustments necessary, hoping that nothing slips through.

Yes, I do not think adding the big hammer, without actually checking
that it is the _only_ possible solution, is a good idea.

Let me explain a bit more something I mentioned in some previous email
in this list, i.e. the idea of "use cases".
In a Debian packaging, there are many things, small and big, that need
to be set/added to make sure a packaging works and is effective. Since
the distribution and its tool evolve (for the better!), then a
packaging is never "done for good", with even very simple packages
needed even once in a while. dh sequencer, multi-arch, R-R-R,
cross-building, etc etc are examples of things added in the more or
less recent past or even still on-going, and which sometime require
changes in out packages. Hence, IMHO, _any_ kind of change ought to
satisfy one or more "use cases", and thus be able to answer all the
following:
- what is the problem?
- how does the proposed change solve the issue?
- how can be the proposed use case be tested
- does the proposed change conflict with any other existing use case?
Being able to answer all the questions above makes sure that:
- we know why a change was done
- what was the exact implementation for a change
- we know how to test that the old use case even in the distant future

> > solve those issues. I think I spotted few issues happened during the
> > last 5.19 update, so I'll mention what I saw and what I suggest as
> > possible solution(s).
> > 
> > 1) kscreenlocker ABI break (#974538)
> > Apparently the kscreenlocker library had an API/ABI break without an
> > SONAME bump, and this was not noticed. A good step was adding a symbols
> > file for the library [1], but it had one issue: it was added based on
> > the _new_ version of the library, so any ABI change was simply not
> > spotted. Easy to solve: when adding a symbols file for a library that
> > does not have one, first add the symbols file for the _existing_
> > version of the library in Debian, then update it to the new version.
> 
> But this doesn't apply for the 5.20 case, right, since we have already
> 5.19 and would see changes?

Well, it depends on how kscreenlocker changed in 5.20 :)
Did you check it so far?

> > (b) happens because of the coinstallability of the private packages
> > with different SONAMEs; while for the majority of libraries this is not
> > an issue, it is in case the old and the new can be indirectly used at
> > the same time.
> > IMHO a potential solution can be to change the private package shlibs
> > to be versioned:
> >   dh_makeshlibs -Vlibkdecorations2private-5-19
> > (similar to what Sandro did for PIM, more about it later)
> > this should make sure that, on upgrade:
> 
> Ok, but since we are targeting 5.20 for bullseye, we would need to
> - Provide: libkdecorations2private-5-20
> - add the dh_makeshlibs -Vlibkdecorations2private-5-20
> Am I missing something?

IMHO we can apply that already in 5.19, even to ease the future
transition to 5.20. I think it will require to upload at the same time:
- kdecoration with the changed shlibs for the private library
- kwin with the bumped kdecoration B-D

If that's OK for you, I can upload these changes this weekend, when the
buildd queue is slightly more empty (right now there are the
"drop py3.8" rebuilds).

> > My ideas (also based on past experience)
> [..]
> > - follow upstream SONAME bumps when they happen
> 
> That is happening (and has happened for 5.19, afair).
> 
> > - check the changes between versions, and do our own ABI bump (using
> >   ABIManager, see e.g. syndication and libktorrent) in case upstream
> >   broke the ABI without an SONAME bump; of course, ideally we should
> 
> That is the tricky part, due to the convolution of c++ symbols and
> checks on internal versus external symbols. Honestly, I don't have a
> good way, and reading all kind of stuffs from other deb devs about it,
> it seems using symbols files for c++ sources are not really
> recommendable.

We have good tooling for this already:
https://qt-kde-team.pages.debian.net/symbolfiles.html
What I consider "the other half" of this tooling is... manual checking
the differences between the old version (5.19.x) and the new one
(5.20.x). Yes, it's a manual work, but it easily allows to spot these
issues, as even symbols files do not catch all the possible ABI
incompatibilities, like reordering virtual methods, adding/removing
private members in public classes, etc -- see also the "cannot" part in
https://community.kde.org/Policies/Binary_Compatibility_Issues_With_C%2B%2B

> > - check for potential behaviour changes without an ABI change
> > Yes, this needs a lot of care, and we need to do that exactly because
> > they are public, so we don't want to unnecessarily break any non-Plasma
> > user of them. IMHO the proposed plasma-base solution is a way too big
> 
> Honestly, I don't see the need to go to that great length. If upstream
> breaks things like this, then downstream has to adapt, too.

Sorry, but sadly we do. If upstream breaks the ABI without bumping the
SONAME, the uploading the new version is definitely not acceptable, and
will break things.

> I just say "poppler" ... (breakage at every release, without bumps).

Bad example: poppler always bumps SONAME together with its API and ABI
of the core (private) library.

> > Regarding (2), IMHO a potential solution is to use what Sandro did for
> > PIM, i.e. have versioned shlibs for the libraries, so when you compile
> > against the new version you get the new version dependency too.
> > The only drawback of this approach is that I see a lot of code
> > copy/pasted in many sources, so having it factored out on a makefile
> > snippet (like library-packages.mk does) would be nicer. OTOH, there are
> > not too many Plasma sources with libraries in this category, so even
> > without a code factoring it would not be that much of copy/paste.
> 
> I have no idea how this was done, and will most probably not implement
> it.

Sandro described it on the list:
https://alioth-lists.debian.net/pipermail/pkg-kde-talk/2020-November/003171.html
One example (they are more or less the same everywhere):
https://salsa.debian.org/qt-kde-team/kde/kontactinterface/-/commit/017b35d1652c067a38d332010fb3066ca39d0fbe

> > To be explicit: I'd prefer to not adopt the proposed
> > https://salsa.debian.org/qt-kde-team/pkg-kde-tools/-/merge_requests/5
> > and the plasma-base usage for now.
> 
> So let us return to the question I posed above: What do you suggest
> *now* - and that means in the remaining short time we have?

I think I already mentioned them, between my previous email and this
one. If not, let me know and I can try to explain it further, or even
directly act.

-- 
Pino Toscano
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part.
URL: <http://alioth-lists.debian.net/pipermail/pkg-kde-talk/attachments/20201207/0d5c1788/attachment.sig>