[Pkg-zfsonlinux-devel] zfs-dkms and Intel QAT

[Chandler]

Hello, we have a computer running bullseye with an Intel 8970 (C62x) PCIe card in it.  I previously had this working fine with ZFS 0.8.2 I think, which I built and installed from the source.  Obviously that was very old, along with all the other software and OS on the machine.  I've now upgraded everything.

Plus I learned from another user I could just export ICP_ROOT to point to the Intel drivers and install your zfs-dkms package, so I have zfs from bullseye-backports installed.  This worked, but only until I rebooted the computer.  Now I can't for the life of me get ZFS to use the QAT anymore, and he hasn't been able to either for over a year it seems.  You might see our lengthy discussions on the ZoL Github pages.

Anyway, I'm pretty sure the problem now is because the ZFS module is getting loaded early in the boot process, before the Intel QAT modules and firmware are loaded.  For example, I see this:

ZFS: Loaded module v2.1.7-1~bpo11+1, ZFS pool version 5000, ZFS filesystem version 5

showing up in the boot messages even before "Starting Install ZFS kernel module..." service.  Why is that and where is that coming from?

I have tried so many things to get the QAT modules loaded before this:
- Both the other user and I tried adding this override:
# cat /etc/systemd/system/zfs-load-module.service.d/override.conf
[Unit]
After=qat_service
(also tried just qat.service since that is what calls the qat_service)

- I have tried `systemctl edit --full zfs-load-module.service` with "Requires=qat.service" and "After=qat.service"
(also tried putting a symlink to /lib/systemd/system/qat.service in /etc/systemd/system/zfs-load-module.service.requires/)

- Then I tried fooling around with other services trying to add requirements even earlier in the boot process, but that usually resulted in loops and services being removed from starting.

- I tried adding a /etc/modules-load.d/intel_qat.conf

- I tried adding a /etc/modprobe.d/intel_qat.conf with something like `softdep zfs pre: qat_c62x`.  soft is not good enough I need something hard!

- I tried adding the qat modules to /etc/modules and /etc/initramfs-tools/modules and updating the initramfs.

- I even figured out how to add a hook in /usr/share/initramfs-tools/hooks/ which didn't break anything, but didn't work either even though `update-initramfs -uv` clearly added qat_c62x,qat_api,intel_qat and /lib/firmware/qat_c62x*.bin to the image

+ more!

Nothing worked!  How frustrating!  I've spent all morning and afternoon trying to get this and have had it "up to here"!!  I'm sure our computer is sick of being rebooted too.  So I would like to know if you have any other suggestions to permanently fix this, at least until the next ZFS or QAT release!

Best Regards,
Chandler