LLVM PPA: Script source availability

Mon Feb 21 15:24:15 GMT 2022

On Mon, Feb 21, 2022 at 3:57 PM Sylvestre Ledru
<sylvestre.ledru at gmail.com> wrote:
>
> Le 21/02/2022 à 15:18, Whisperity a écrit :
> > In addition: I am specifically looking for the people/community
> > maintaining the "nightly" APT.LLVM.ORG and **not** the official
> > release Debian package repository.
> > I might be sending mails to the wrong place?
>
> This is the correct place but I am almost the only one doing the maintenance of both.
> apt.llvm.org is the nightly stuff of what I am uploading in Debian & ubuntu.
> They are really the same.
>
> > I was working off of this information:
> >
> >> $ apt-cache show clang-12 | grep 'Version\|Maintainer'
> >> Version: 1:12.0.1~++20211216045758+fed41342a82f-1~exp1~20211216170241.12
> >> Maintainer: LLVM Packaging Team <pkg-llvm-team at lists.alioth.debian.org>
> >
> > On Mon, Feb 21, 2022 at 3:14 PM Whisperity <whisperity at gmail.com> wrote:
> >>
> >> Dear Sylvestre,
> >>
> >> I understand, but these are almost ten-year-old threads at the moment.
>
> Sorry but what do you think changed in 10 years in the Debian and Ubuntu packaging world ? ;)
> (not trolling, I am just saying that most of the fundamentals are still the same)

The idea that you would expect ABI/API compatibility between versions
of the same command, and that it is inherently bad to offer the choice
to the user without breaking the default behaviour.
That's what I could gather that the issue with changing the version
behind "clang" via update-alternatives would result in issues from
that. While certainly it *could*, it is only an issue if the "default"
supplied version changes. At which point, we are in the same issue as
if someone upgraded distribution: suddenly, `gcc` means 9.0 or 10.0,
not 7.5. But their scripts only calling `gcc` by default SHOULD work.
(And it is generally not the maintainers' problem if someone's
third-party script simply does not, right?)

> >> Are we sure the state of the world did not change at all since, and
> >> the arguments there (already contested by others) are still valid?
> >> 'llvm-default' or 'llvm-defaults' does not even turn up any packages
> >> at this point (on Ubuntu 18.04 or Debian 10).
> I don't remember a time for which llvm-defaults installed anything.
>
> llvm-defaults provides the clang, lldb, lld, etc symlink to point a specific version.
> Example:
> https://packages.debian.org/unstable/clang
> https://packages.debian.org/unstable/lld

Ah, it is a **source package**, that's why I was not finding anything.
I was trying to look it up with `apt-cache show` or `aptitude` on both
of my machines.

>
> If you want to use a specific version of clang, run
> CC=clang-13 ./configure
> (or the equivalent).
>
> What is wrong this that solution?

People don't usually use the compiler purely for compiling. The script
and the build system would need to *understand*, through the entire
pipeline, that environment variable. Most build systems do, but the
moment you have a custom script or something, it won't.
If people are working off of their history, and their history
contained "clang-11" but suddenly it is not there, their scripts or
commands won't work.

Certainly, it would be easier to explain to people how versions and
releases work, but as I said earlier, in many cases, unfortunately,
this is not a "competence" prerequisite we could meaningfully enforce.

Consider the following two discussions. Similar to the mention in the
original linked threads where people said that certain tools and such
expect to only have "llvm-config" and Clang, and various other things
in the right locations, as opposed to having the ability to configure
everything neatly. It of course could be the mistake of those projects
and not the package maintainers here!

We use clang, and various other LLVM tools, not through a build
system, but for visualisation, debugging, transformations, fact
extraction, etc.
Which means in many cases we would need to implement recovering data
from environment variables, or deal with the specifics of this or that
distribution.
Compared to that, relying only on "PATH" and the user adding the right
binary release to their PATH *generally* works.
What we would need here, is the ability to manage the "system level
PATH" properly for new versions.

[Current workflow without update-alternatives and having to deal with suffixes]

S(ysadmin): *uploads newer LLVM release, suddenly clang-15 is the
version available*
U(ser): *tries to run commands from history or from scripts that have
"clang-13", or purely "clang" in them*
U: Why are my scripts not working, it keeps saying command not found?
S: We updated versions, you need to replace it.
U: *replaces in certain places only, because honest mistake, runs
script, it fails after some hours of work*
U: What?! *<curated list of expletives aimed at sysadmin>*
S: *<curated list of expletives aimed at both apt/debian and the user>*
U: *does some more replacing, hopefully not missing anything this time*
*It worked.*

[With update-alternatives]

S: *installs new release, sets system-wide alternative to new version*
S: *may or may not uninstall old version*
U: *all scripts use "clang", "clang++", "ld.lld", whatever*
U: *runs their scripts or commands to produce a new build*
*It works. Everyone is happy.*

The default meaning of "clang" would as expected, I would of course
want to stay the same, whichever version the distribution is at right
now.

>
> Now, if you have something ready for a MR, feel free to submit it. I will be happy to have a look to it.

Not yet. But now that I know where the repository lies, I hope I can
find the time to read it through and understand it.
What I did on the aforementioned machine I administer was just to run
calls like this:

~~~~~
17305  sudo update-alternatives --install /usr/local/bin/clang clang
/usr/bin/clang-12 12 --slave /usr/local/bin/clang++ clang++
/usr/bin/clang++-12 --slave /usr/local/bin/asan_symbolize
asan_symbolize /usr/bin/asan_symbolize-12 --slave
/usr/local/bin/clang-cpp clang-cpp /usr/bin/clang-cpp-12
17306  sudo update-alternatives --install /usr/local/bin/lld lld
/usr/bin/lld-12 12 --slave /usr/local/bin/ld.lld ld.lld
/usr/bin/ld.lld-12
17309  sudo update-alternatives --install /usr/local/bin/clangd clangd
/usr/bin/clangd-12 12
17310  sudo update-alternatives --install /usr/local/bin/clang-format
clang-format /usr/bin/clang-format-12 12 --slave
/usr/local/bin/clang-format-diff clang-format-diff
/usr/bin/clang-format-diff-12 --slave /usr/local/bin/git-clang-format
git-clang-format /usr/bin/git-clang-format-12
17311  sudo update-alternatives --install /usr/local/bin/clang-tidy
clang-tidy /usr/bin/clang-tidy-12 12 --slave
/usr/local/bin/clang-tidy-diff.py clang-tidy-diff.py
/usr/bin/clang-tidy-diff-12.py --slave /usr/local/bin/run-clang-tidy
run-clang-tidy /usr/bin/run-clang-tidy-12 --slave
/usr/local/bin/run-clang-tidy.py run-clang-tidy.py
/usr/bin/run-clang-tidy-12.py
~~~~~

I gathered the list of binaries simply by inspecting what the related
binary Debian package copies onto the system.

All I would like is automate running these commands (with the right
version number added and the list of relevant binaries automatically
generated) at first glance in the "install" or "postinst" script,
similarly to how "java-common-0.71//debian/java-common.install"
calling "java-common-0.71//scripts/update-java-alternatives" does it.

>
> >> For example, someone already mentioned Java there, and indeed (taken
> >> from an Ubuntu 18.04.6 LTS system):
> >>
> >>> ╰─ update-alternatives --query java
> >>> Name: java
> >>> Link: /usr/bin/java
> >>> Slaves:
> >>>   java.1.gz /usr/share/man/man1/java.1.gz
> >>> Status: auto
> >>> Best: /usr/lib/jvm/java-11-openjdk-amd64/bin/java
> >>> Value: /usr/lib/jvm/java-11-openjdk-amd64/bin/java
> >>>
> >>> Alternative: /usr/lib/jvm/java-11-openjdk-amd64/bin/java
> >>> Priority: 1111
> >>> Slaves:
> >>>   java.1.gz /usr/lib/jvm/java-11-openjdk-amd64/man/man1/java.1.gz
> >>>
> >>> Alternative: /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java
> >>> Priority: 1081
> >>> Slaves:
> >>>   java.1.gz /usr/lib/jvm/java-8-openjdk-amd64/jre/man/man1/java.1.gz
> >>
> >> There were a lot of changes from Java 8 to 11 on the command-line
> >> interface which makes the two incompatible. (Although most of those
> >> are not everyday-used flags.)
> Yeah, Matthias (who particpated to the discussions on launchpad) and I (at some point)
> have been maintaining openjdk packages too.
>
> It also uses a similar approach to llvm-defaults:
> https://packages.debian.org/buster/default-jre (and others)
>
>
> Cheers,
> Sylvestre
>