Bug#910664: Acknowledgement (ghc: ghc package can no longer be cross-compiled)

[Helmut Grohne]

Hi Ilias,

On Mon, Dec 11, 2023 at 10:08:49PM +0200, Ilias Tsitsimpis wrote:
> On Sun, Dec 10, 2023 at 09:01PM, Helmut Grohne wrote:
> > First and foremost, ghc now actively refuses being cross build with an
> > $(error ...). Would you mind weakening this to a $(warning ...)? While
> > that you don't want to support cross building of ghc, would you mind
> > others (like me) supporting it? Yes, it would still fail, but then
> > https://crossqa.debian.net/src/ghc could show a more useful reason for
> > that failure.
> 
> The reason I have this as an $(error ...) is because the new Hadrian
> build system doesn't support cross-compiling GHC [1]. This is not a
> limitation of our Debianization (i.e., it's not that we refuse to
> cross-build GHC, we *cannot* cross-build GHC), this is upstream
> limitation. Given that, I believe $(error ...) is more appropriate here
> than $(warning ...).
> 
> [1] https://gitlab.haskell.org/ghc/ghc/-/issues/23975

I'm not yet entirely convinced that it cannot work, but I certainly
agree that it does not work now. However, the $(error ...) completely
hides the reason. I'd like to see a cross build failure that shows why
this cannot work and that failure only comes about much later. What
advantage do you see in hiding this failure from QA systems?

> The resulting stage1 tools are a cross-compiler, not the final
> cross-compiled binaries (this is why they are prefixed with the host gnu
> triplet). I am not sure how we will handle this, I am waiting to see on
> what upstream will do to support cross-building GHC. For now, I believe
> applying the attached patch doesn't help.

To me, this looks fairly obvious. The stage1 tools are not installed
into any .deb at any time. What we install into a .deb is the stage2.
All that we need stage1 for is running it during build. This is where my
patch helps. And quite objectively, the patch helps in the sense that
the build goes further once you apply it.

> GHC does *not* use the same terminology as Debian. GHC requires that
> HOST and BUILD are the same. You can read more about this here [2],
> though keep in mind this document is severely outdated with the Hadrian
> build system.
> 
> [2] https://gitlab.haskell.org/ghc/ghc/-/wikis/building/cross-compiling#terminology-and-background

That page carefully explains that it uses the same terminology as
Debian. It also says that host and build must equal which translates to
"cross compilation is not supported at all", because the definition of a
cross build is build and host being different from one another.

However, that page also goes on and says that stage2 is being built
using stage1. Since stage1 is a cross compiler, stage1 runs on the build
machine and hence we are building stage2 on our build machine. And since
stage2 runs on the target, the stage2 build process effectively has
build differ from host contradicting what the page says earlier. So
indeed, something is wrong here. From my pov, the information that most
likely is wrong is that ghc cannot be cross compiled.

> I believe we need to work with upstream to add support for
> cross-compiling GHC to the new Hadrian build system. As explained here [3],
> this is currently not possible. I tested everything I could think of,
> but I don't see how we can move forward without reworking how the
> Hadrian build system works. Upstream has started working on this, but
> it's moving slowly [4].

I agree that we need to work with upstream. However, we also should make
it work on the Debian side as good as possible, but currently it aborts
early saying that this cannot work. Can we move incrementally and enable
as much as works?

> In summary, as a result of the switch to the Hadrian build system, we
> are now unable to cross-compile GHC. Since this issue now has all the
> latest context, I propose we keep it open and work here on
> cross-building GHC.

The Debian build does not get far enough to demonstrate this failure.
Talking to upstream is much easier if you can point them at failing
build logs and we currently cannot, because we are still facing issues
before even running Hadrian.

Regarding that --build flag, I now see that we actually run configure
twice. Connecting what you said earlier, I guess that one of those
invocations is for the stage1 compiler and the second one is for the
stage2 compiler though I may be wrong. If that guess is correct, then
yes, the stage1 compiler definitely needs --host=$(DEB_BUILD_GNU_TYPE),
but doing it like that still passes host architecture compiler flags and
you end up running e.g. aarch64-linux-gnu-gcc -fcf-protection or
x86_64-linux-gnu-gcc -mbranch-protection=standard and fail either way.
So the way to pass that --host flag is:

    dpkg-architecture -a$(DEB_BUILD_ARCH) -f -c dh_auto_configure --reload-all-buildenv-variables -- $(CONFIGURE_ARGS)
    # CONFIGURE_ARGS contains neither --build nor --host, but it contains --target

Then when we build stage2, we are using stage1 and there we probably
should not be passing --host=$(DEB_BUILD_GNU_TYPE). My guess now is that
this is where that wiki page is wrong.

Given all of the above, I still think that my patch moves us forward,
because it allows https://crossqa.debian.net/src/ghc display a real
failure that we can use in communication with upstream and because
calling stage1 tools in the right way pretty much is required to seeing
the later failure from using Hadrian and being able to work on that
aspect. This failure is not very useful to look at:
https://crossqa.debian.net/build/ghc_9.4.7-1_amd64_20231029134737.log

Is this reasoning good enough to convince you that we do have options
for improving the status quo?

Helmut