Bug#841923: AW: AW: Bug#841923: libclang-common-3.9-dev: missing multilib binaries

Tue Nov 1 21:54:08 UTC 2016

> I don't think this is reasonable leave it as it.
> I would like to see this changes in 3.8 and this will impact too much
> Debian.
I am not saying to leave it, but to do the bigger change in smaller
steps. You seem to miss, that not only the amd64 package will change,
but any host architecture will aswell. I cant test anything but amd64,
and we might mess thing up for the rest of them, possibly without
noticing

** Example, of what I believe will happen (only for arm64. mips64,
ppc64, sparc64? likewise):
* Now:

arm64 package contains:
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-aarch64.a
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-aarch64.a.syms

armhf package contains:
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armhf.a
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armhf.a.syms

armel package contains (would contain on a jessie backport, just my guess):
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armel.a
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armel.a.syms

* After adding the g++-multilib dependency:
arm64 package contains:
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-aarch64.a
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-aarch64.a.syms
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armhf.a
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armhf.a.syms
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armel.a
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armel.a.syms

armhf package contains:
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armhf.a
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armhf.a.syms
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armel.a
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armel.a.syms

armel package contains (would contain on a jessie backport, just my guess):
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armel.a
/usr/lib/llvm-3.9/lib/clang/3.9.0/lib/linux/libclang_rt.ubsan_standalone-armel.a.syms

** this will affect packaging, and all this is just a GUESS now. We
would know ALOT better what we need to do if we let the build server
run once.
If we start moving around files, we might miss some, or break the build.

> $ debdiff  /tmp/libclang-common-3.8-dev_3.8.1-12_amd64.deb
> libclang-common-3.8-dev_3.8.1-13_amd64.deb
As said, amd64 is just one architecture, I am talking about all of them.

> You are talking about libclang_rt.asan-i386.so and libclang_rt.asan-i686.so,
> right?
These are the ones that add the new lib32* dependencies, and the ones
that are troublesome for dh_shlibdeps

What would you say, if for now we just remove the dependencies to the
lib32* libraries?
You would have to adjust the filter depending on the architecture
(thats exactly what I dont know to figure out easily for other archs).
eg.:
dh_shlibdeps -l/buildllvm/llvm-toolchain-3.9-3.9/debian/tmp//usr/lib/llvm-3.9/lib/
-Xlibclang_rt.asan-i686.so -Xlibclang_rt.asan-i386.so

I added a patchv2 for just this. To me this makes alot of sense anyway
since you dont strictly depend on the host toolchain C/C++ libraries.
For example I am using clang with custom toolchains in the /opt
directory and the C/C++ libraries are picked up from there.
They might make sense as recommendations.

For splitting the archives, I`d like to write down a scheme, but this
will take a few days. The patchv2 should result in the same
dependencies as before, but still have the multilibs included.
Works correctly on amd64 and should compile on all others, some other
archs might end up with additional dependencies (32/64bit C/C++libs)
like the previous patch - which we can fix by expanding the table of
extensions (MULTILIB_EXTS).

Kind Regards,
Norbert

PS.: an older patch for splitting up the libs is attached by Bug
#829441. But I think a different approach would be better


2016-11-01 21:24 GMT+01:00 Sylvestre Ledru <sylvestre at mozilla.com>:
> Le 01/11/2016 à 19:56, Norbert Lange a écrit :
>>
>> Hi,
>>
>> we absolutely should do this. I believe we have some communication
>> problems, because I brought this up multiple times.
>
> Probably me, sorry :/
>>
>> I am not sure how to solve it, I can think of multiple ways. But it
>> would help if you just apply this path as it is, and let it build for
>> the ~10 architectures. Can you do this somehow, maybe just keep it in
>> experimental?
>
> I don't think this is reasonable leave it as it.
> I would like to see this changes in 3.8 and this will impact too much
> Debian.
>
> So, we should find a proper solution.
> However, I "only" saw the i386 files, not armel or others.
> What should be the result on arm archs?
>
>> First, it helps if we know we start with a working build (on all
>> platforms) before modifying it, and which libraries would normally be
>> built.
>> Then I would like to be able to make a list of libraries for all
>> architectures, since I believe this will differ alot.
>
>
> $ debdiff  /tmp/libclang-common-3.8-dev_3.8.1-12_amd64.deb
> libclang-common-3.8-dev_3.8.1-13_amd64.deb
> [The following lists of changes regard files as different if they have
> different names, permissions or owners.]
>
> Files in second .deb but not in first
> -------------------------------------
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.asan-i386.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.asan-i386.so
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.asan-i686.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.asan-i686.so
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.asan-preinit-i386.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.asan-preinit-i686.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.asan_cxx-i386.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.asan_cxx-i686.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.builtins-i386.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.builtins-i686.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.cfi-i386.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.cfi-i686.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.cfi_diag-i386.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.cfi_diag-i686.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.profile-i386.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.profile-i686.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.safestack-i386.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.safestack-i686.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.ubsan_standalone-i386.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.ubsan_standalone-i686.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.ubsan_standalone_cxx-i386.a
> -rw-r--r--  root/root
> /usr/lib/llvm-3.8/lib/clang/3.8.1/lib/linux/libclang_rt.ubsan_standalone_cxx-i686.a
>
> this could be the opportunity to move them into a (or several) dedicated
> packages.
>
> So, we could create:
> libclang-sanitizer => with the libraries for the arch
> and
> libclang-sanitizer-multilib => with the libraries for the other supported
> archs (i386 for amd64, arm* for other I guess)
>
>  I don't think we can use some voodoo-multiarch magic here :/
>
>> Also (I brought this up before): I dont know if the shared sanitizer
>> libraries are actually used anywhere. The static libraries dont make
>> problems, so if we can drop the shared ones then this is one problem
>> solved.
>
> You are talking about libclang_rt.asan-i386.so and libclang_rt.asan-i686.so,
> right?
>
> S
>
-------------- next part --------------
diff -burN debian.org/control debian/control

--- debian.org/control	2016-10-31 10:47:52.000000000 +0100
+++ debian/control	2016-11-01 22:12:16.996263027 +0100
@@ -7,7 +7,7 @@
     cmake, perl, libtool, chrpath, texinfo, sharutils, libffi-dev (>= 3.0.9),
     lsb-release, patchutils, diffstat, xz-utils, python-dev,
     libedit-dev, swig, python-six, python-sphinx, ocaml-nox, binutils-dev,
-    libjsoncpp-dev,
+    libjsoncpp-dev, g++-multilib,
     lcov, procps, help2man, dh-ocaml, zlib1g-dev
 Build-Conflicts: oprofile, ocaml, libllvm-3.4-ocaml-dev, libllvm-3.5-ocaml-dev,
   libllvm-3.8-ocaml-dev, libllvm-3.9-ocaml-dev
diff -burN debian.org/rules debian/rules
--- debian.org/rules	2016-10-31 10:47:52.000000000 +0100
+++ debian/rules	2016-11-01 22:29:16.352794644 +0100
@@ -21,6 +21,7 @@
 DEB_HOST_GNU_TYPE   = $(shell dpkg-architecture -qDEB_HOST_GNU_TYPE)
 DEB_HOST_ARCH_BITS  = $(shell dpkg-architecture -qDEB_HOST_ARCH_BITS)
 DEB_HOST_ARCH       = $(shell dpkg-architecture -qDEB_HOST_ARCH)
+DEB_HOST_GNU_CPU    = $(shell dpkg-architecture -qDEB_HOST_GNU_CPU)
 
 OCAML_STDLIB_DIR    ?= $(shell ocamlc -where)
 
@@ -29,6 +30,14 @@
 CONFIGURE_EXTRA =
 CMAKE_EXTRA =
 
+ifeq ($(DEB_HOST_GNU_CPU),x86_64)
+MULTILIB_EXTS := i386 i686 
+else ifneq (,$(filter i%86,$(DEB_HOST_GNU_CPU))
+MULTILIB_EXTS := x86_64
+else
+MULTILIB_EXTS :=
+endif
+
 ifneq (,$(filter $(DEB_HOST_ARCH),powerpc powerpcspe))
 LDFLAGS_EXTRA += -latomic
 endif
@@ -400,7 +409,7 @@
 
 
 override_dh_shlibdeps:
-	LD_LIBRARY_PATH=$$LD_LIBRARY_PATH:$(DEB_INST)/usr/lib/llvm-$(LLVM_VERSION)/lib/ dh_shlibdeps
+	dh_shlibdeps $(patsubst %,-Xlibclang_rt.asan-%.so,$(MULTILIB_EXTS))
 
 override_dh_installman:
 	dh_installman
