Bug#1082816: petsc: enable AMD GPU support

[Cordell Bloor]

Source: petsc
Version: 3.20.6+dfsg1-3
Severity: wishlist
X-Debbugs-Cc: cgmb at slerp.xyz

Dear Maintainer,

The petsc toolkit has an AMD ROCm backend for hardware acceleration on
AMD GPUs. It would be good to enable this functionality. The AMD backend
depends on the following packages:

hipcc
libamdhip64-dev
libhipblas-dev
libhiprand-dev
libhipsparse-dev
libhipsolver-dev
librocsparse-dev
librocsolver-dev
librocblas-dev
librocrand-dev
librocthrust-dev
librocprim-dev

The petsc toolkit may require kokkos with ROCm support, which has been
requested in Debian Bug #1082813 [1].

The ROCm libraries packaged for Debian are currently built for gfx803,
gfx900, gfx906, gfx908, gfx90a, gfx1010, gfx1030, gfx1100, gfx1101,
gfx1102. They will soon be adding gfx942. However, kokkos appears to be
limited to gfx900, gfx906, gfx908, gfx90a, gfx940, gfx1030, and gfx1100.
I supppose petsc may be limited to the kokkos-supported architectures.

In controlling the ROCm portion of the petsc build, relevent
configuration flags include --with-hip-arch=<arch>, HIPPPFLAGS=<flags>,
and --with-hip-lib=<libs>.

I would suggest adding -DROCPRIM_USE_ARCH_CONVERSION to the
DEB_CXXFLAGS_MAINT_PREPEND flags when building petsc. This is a rocprim
build option that affects rocthrust. It causes gfx902, gfx909, and
gfx90c to use gfx900 code paths and causes gfx1031, gfx1032, gfx1033,
gfx1034, gfx1035, and gfx1036 to use gfx1030 code paths. It may or may
not be sufficient for petsc support to those architectures, but it will
at least be necessary.

If you require access to compatible hardware to test this functionality,
please reach out privately and I may be able to help.

Sincerely,
Cory Bloor

[1]: https://bugs.debian.org/1082813

-- System Information:
Debian Release: trixie/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 6.10.11-amd64 (SMP w/32 CPU threads; PREEMPT)
Locale: LANG=C, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: unable to detect