Bug#916883: nvidia-kernel-dkms:amd64: does not build for x86_64 kernel on i386 userspace with kernel 4.19

[Andreas Beckmann]

Receipe for reproduction:

amd64 host
i386 pbuilder chroot with sid+testing sources
dpkg --add-architecture amd64
apt-get update
apt-get install linux-headers-4.18.0-3-amd64 linux-headers-4.19.0-1-amd64 module-assistant
apt-get install nvidia-kernel-source:amd64 nvidia-legacy-340xx-kernel-source:amd64

sh /usr/share/doc/nvidia-legacy-340xx-kernel-source/build-module-packages.sh
sh /usr/share/doc/nvidia-kernel-source/build-module-packages.sh

the legacy driver builds fine, the current driver fails with:

ld  -r -o /usr/src/modules/nvidia-kernel/nvidia/nv-interface.o /usr/src/modules/nvidia-kernel/nvidia/nv-frontend.o /usr/src/modules/nvidia-kernel/nvidia/nv-instance.o /usr/src/modules/nvidia-kernel/nvidia/nv-gpu-numa.o /usr/src/modules/nvidia-kernel/nvidia/nv.o /usr/src/modules/nvidia-kernel/nvidia/nv-acpi.o /usr/src/modules/nvidia-kernel/nvidia/nv-chrdev.o /usr/src/modules/nvidia-kernel/nvidia/nv-cray.o /usr/src/modules/nvidia-kernel/nvidia/nv-dma.o /usr/src/modules/nvidia-kernel/nvidia/nv-gvi.o /usr/src/modules/nvidia-kernel/nvidia/nv-i2c.o /usr/src/modules/nvidia-kernel/nvidia/nv-mempool.o /usr/src/modules/nvidia-kernel/nvidia/nv-mmap.o /usr/src/modules/nvidia-kernel/nvidia/nv-p2p.o /usr/src/modules/nvidia-kernel/nvidia/nv-pat.o /usr/src/modules/nvidia-kernel/nvidia/nv-procfs.o /usr/src/modules/nvidia-kernel/nvidia/nv-usermap.o /usr/src/modules/nvidia-kernel/nvidia/nv-vm.o /usr/src/modules/nvidia-kernel/nvidia/nv-vtophys.o /usr/src/modules/nvidia-kernel/nvidia/os-interface.o /usr/src/modules/nvidia-kernel/nvidia/os-mlock.o /usr/src/modules/nvidia-kernel/nvidia/os-pci.o /usr/src/modules/nvidia-kernel/nvidia/os-registry.o /usr/src/modules/nvidia-kernel/nvidia/os-usermap.o /usr/src/modules/nvidia-kernel/nvidia/nv-modeset-interface.o /usr/src/modules/nvidia-kernel/nvidia/nv-pci-table.o /usr/src/modules/nvidia-kernel/nvidia/nv-kthread-q.o /usr/src/modules/nvidia-kernel/nvidia/nv-kthread-q-selftest.o /usr/src/modules/nvidia-kernel/nvidia/nv-memdbg.o /usr/src/modules/nvidia-kernel/nvidia/nv-ibmnpu.o /usr/src/modules/nvidia-kernel/nvidia/nv-report-err.o /usr/src/modules/nvidia-kernel/nvidia/nv_uvm_interface.o /usr/src/modules/nvidia-kernel/nvidia/nvlink_linux.o
ld: relocatable linking with relocations from format elf64-x86-64 (/usr/src/modules/nvidia-kernel/nvidia/nv-frontend.o) to format elf32-i386 (/usr/src/modules/nvidia-kernel/nvidia/nv-interface.o) is not supported
make[4]: *** [/usr/src/modules/nvidia-kernel/nvidia/nvidia.Kbuild:95: /usr/src/modules/nvidia-kernel/nvidia/nv-interface.o] Error 1
make[4]: *** Waiting for unfinished jobs....
...
make[3]: *** [/usr/src/linux-headers-4.19.0-1-common/Makefile:1532: _module_/usr/src/modules/nvidia-kernel] Error 2
make[3]: Leaving directory '/usr/src/linux-headers-4.19.0-1-amd64'
make[2]: *** [Makefile:146: sub-make] Error 2
make[2]: Leaving directory '/usr/src/linux-headers-4.19.0-1-common'
make[1]: *** [Makefile:79: modules] Error 2
make[1]: Leaving directory '/usr/src/modules/nvidia-kernel'
make: *** [debian/rules:47: build-stamp] Error 2

Why don't they fail both?
OK, they use completely different ld invocations:

/usr/src/modules/nvidia-legacy-340xx-kernel/nvidia-modules-common.mk:   @$(LD) $(EXTRA_LDFLAGS) $(MODULE_COMMON_SCRIPT) -r -o $(KERNEL_GLUE_NAME) $(KERNEL_GLUE_OBJS)

/usr/src/modules/nvidia-kernel/Makefile:        $(LD) $(NV_MODULE_COMMON_SCRIPT) -r -o $@ \
/usr/src/modules/nvidia-kernel/nvidia-modeset/nvidia-modeset.Kbuild:    $(LD) -r -o $@ $^
/usr/src/modules/nvidia-kernel/nvidia/nvidia.Kbuild:    $(LD) -r -o $@ $^


On 2018-12-19 22:53, Tomas Janousek wrote:
> Linux 4.19 renamed LDFLAGS to KBUILD_LDFLAGS [1], so use-kbuild-flags.patch
> doesn't work any more. I can confirm that using KBUILD_LDFLAGS fixes the
> issue. Do note that older kernels don't set KBUILD_LDFLAGS, so we need to
> support both for some time.
s/for some time/forever/

probably: $(or $(KBUILD_LDFLAGS),$(LDFLAGS))

That turns the failing command back into
ld -m elf_x86_64  -z max-page-size=0x200000 -r -o /usr/src/modules/nvidia-kernel/nvidia/nv-interface.o /usr/src/modules/nvidia-kernel/nvidia/nv-frontend.o ...

> Oh, and why does https://salsa.debian.org/nvidia-team/nvidia-graphics-drivers
> need signing in? :-(

Because the repository is not yet at that location.


Andreas