Bug#1078010: libnet-z3950-zoom-perl FTCBFS: hard codes the build architecture pkg-config

[Helmut Grohne]

Hi Gregor,

On Tue, Aug 06, 2024 at 06:27:19PM +0200, gregor herrmann wrote:
> I played around a bit:
> - crosscompiled the package as is from amd64->i386 (with pbuilder's
>   --host-arch) and it failed
> - created a patch similar to [0] using ExtUtils::PkgConfig [1]

I believe our goal should be to somehow make that patch of yours work,
because it is a natural way to write it and I expect other upstreams to
do it that way. To me, this looks like the better solution even though
it does not work right now. If you use that, you don't fix the FTCBFS,
but you push the problem elsewhere and it no longer is
libnet-z3950-zoom-perl that needs changes to fix it. ;)

> - crossbuilding fails with:
> 
>   The following packages have unmet dependencies:
>    builddeps:/build/libnet-z3950-zoom-perl_1.32-1.dsc:i386 : Depends: debhelper-compat:i386 (= 13)
>                                                              Depends: libextutils-pkgconfig-perl:i386 but it is not installable
>   E: Unable to correct problems, you have held broken packages.
> 
> Hm, does libextutils-pkgconfig-perl (arch:all) need "Multi-Arch: foreign"?

That is the right way of looking at it. Let me also explain how to
answer your question. Do you expect libextutils-pkgconfig-perl to behave
in an architecture-independent way?

My answer here is maybe. It depends on how you define it. We want
ExtUtils::PkgConfig->cflags to yield flags for the "host" architecture,
but what "host" means from a libextutils-pkgconfig-perl view is not
entirely clear. It could be that we communicate the architecture to
libextutils-pkgconfig-perl via its dependency and then it certainly
would not be architecture-independent as we'd expect a
libextutils-pkgconfig-perl:amd64 to behave differently from a
libextutils-pkgconfig-perl:i386 one. The other way is that you somehow
pass the architecture at runtime via some variable (explicit or
implicit, argument or global or environment). In that way, the
architecture is part of your input and we expect it to yield the same
flags if given the same architecture and then the answer would be yes.

The other way to look at it is looking at its dependencies. It depends
on pkg-config, which is an architecture-dependent package (because each
pkg-config:$arch provides the $triplet-pkg-config entry point). If
libextutils-pkgconfig-perl uses this entry point, it must be its package
architecture that defines what "host" is as it forwards this
architecture on its dependency. Otherwise, it can only call into
pkg-config via /usr/bin/pkg-config, but then it gets the results for the
build architecture, which isn't what we wanted.

So without further modifications, we cannot just mark it Multi-Arch:
foreign (and that also is the reason why I didn't file a bug asking for
that yet).

Any time we want to use libextutils-pkgconfig-perl in a package build,
we need to ensure that our installation set contains a pkg-config for
the host architecture. No matter how we do it, as long as it remains
Architecture: all, it technically has no way of requesting the host
architecture's pkg-config via its own dependencies, so the only way to
keep it Architecture: all is to state that any package that declares a
build dependency on libextutils-pkgconfig-perl must also declare a build
dependency on pkg-config. The other way is converting it to
Architecture: any + Multi-Arch: same (this conversion is often dubbed
"the multiarch interpreter workaround for the multiarch interpreter
problem"). Once doing this a regular build dependency (which is
implicitly constrained to the host architecture) gets its architecture
constraint passed through to the pkg-config dependency and stuff might
work.

I note that this also poses a very unusual use of the architecture
qualification of the libextutils-pkgconfig-perl module. For most other
perl modules, the expectation of an architecture-qualified dependency is
that you can load the module (and any extension modules that it
transitively depends on) into a perl interpreter of said architecture.
With libextutils-pkgconfig-perl, there are no extension modules (outside
core perl) involved, so this does not matter. Rather the above
interprets the architecture qualification as a means of requesting
package configuration for the qualified architecture. Such an
inconsistency is prone to biting us later, so maybe the route of pushing
the need to depend on pkg-config down to the consumer is the better
choice in the end.

Note that all of this only looks at the packaging metadata only. I
haven't yet implied anything about the implementation of
libextutils-pkgconfig-perl that also likely needs to change to support
such use.

The bottom line roughly is "it's difficult". I hope we can agree with
that.

> If I put ExtUtils::PkgConfig manually into ./ExtUtils/PkgConfig.pm
> and run the Makefile.PL call from above manually with an additional
> -I. I get:
> 
>   # /usr/bin/perl -I. -I/usr/lib/i386-linux-gnu/perl/cross-config-5.38.2 Makefile.PL INSTALLDIRS=vendor "OPTIMIZE=-g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/build/libnet-z3950-zoom-perl-1.32=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2" "LD=i686-linux-gnu-gcc -g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/build/libnet-z3950-zoom-perl-1.32=. -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro -Wl,-z,now"
>   Package yaz was not found in the pkg-config search path.
>   Perhaps you should add the directory containing `yaz.pc'
>   to the PKG_CONFIG_PATH environment variable
>   Package 'yaz', required by 'virtual:world', not found
>   Package yaz was not found in the pkg-config search path.
>   Perhaps you should add the directory containing `yaz.pc'
>   to the PKG_CONFIG_PATH environment variable
>   Package 'yaz', required by 'virtual:world', not found
>   Package yaz was not found in the pkg-config search path.
>   Perhaps you should add the directory containing `yaz.pc'
>   to the PKG_CONFIG_PATH environment variable
>   Package 'yaz', required by 'virtual:world', not found
>   …
> 
> A bit better but /usr/lib/i386-linux-gnu/pkgconfig/yaz.pc is
> apparently not found.

This likely is due to libextutils-pkgconfig-perl not prepending the
toolchain prefix to pkg-config and thus using the build architecture
(likely amd64) one. That's the other piece of the puzzle, we need to
enhance ExtUtils::PkgConfig to somehow pick up the toolchain prefix from
the perl Config.pm and we pass
-I/usr/lib/i386-linux-gnu/perl/cross-config-5.38.2 to supply a config
matching the host architecture. To make matters worse, the toolchain
prefix isn't readily available in there. I suppose the best we could do
is examine cc and strip "-gcc", but that's a hack at best. Possibly perl
needs to record this value or it needs to be passed in some other way.

The bottom line roughly is "it's difficult". I hope we can agree with
that.

> And in the end ExtUtils::PkgConfig just calls `pkg-config`, so any
> prefix would need to be found and added there. Alright, sorry for the
> detour :)

Thank you for the detour. I think you're spot-on. If you want to further
this adventure, I'm happy to continue. Making ExtUtils::PkgConfig just
work would fix a pile of packages and make for fewer patches and thus
less maintenance cost. I'm not looking for quick solutions with
long-term maintenance costs, but for good solutions with low permanent
costs.

> So it looks your patch it will be, unless someone has another idea.

Actually, I suggest that to just take your ExtUtils approach and thus
double down on making this work somehow. This way of combining
domain-specific knowledge from different domain experts that you started
here is exactly how we can make progress on such issues.

Thank you very much

Helmut