I just found my Debian arm64 Pi and tested this, it seems to work after setting "iomem=relaxed" on the kernel command line. More than enough time has passed since my last mail where I said I would NMU after testing, so I have gone ahead with the upload.