Bug#1064704: gtk4: FTBFS: one run of gtk:tools / validate test failed

[Simon McVittie]

Control: tags -1 + confirmed

Context for Mesa maintainers: gtk4 passed its build-time tests on
2024-01-29, but is now failing in a test rebuild. I can reproduce this,
and I think it's a regression triggered by Mesa changes (see also
https://gitlab.freedesktop.org/mesa/mesa/-/issues/10293 upstream).

On Sun, 25 Feb 2024 at 20:37:28 +0100, Lucas Nussbaum wrote:
> > 1332/1515 gtk:tools / validate                                                                                                              FAIL            2.07s   0/9 subtests passed
> > >>> GDK_BACKEND=x11 G_ENABLE_DIAGNOSTIC=0 GTK_QUERY_SETTINGS=/<<PKGBUILDDIR>>/debian/build/deb/tools/gtk4-query-settings MALLOC_PERTURB_=208 ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 GTK_A11Y=test GDK_DEBUG=default-settings GTK_CSD=1 GTK_BUILDER_TOOL=/<<PKGBUILDDIR>>/debian/build/deb/tools/gtk4-builder-tool GSETTINGS_SCHEMA_DIR=/<<PKGBUILDDIR>>/debian/build/deb/gtk UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 TEST_RESULT_DIR=/<<PKGBUILDDIR>>/debian/build/deb/testsuite/tools/output GSETTINGS_BACKEND=memory G_TEST_BUILDDIR=/<<PKGBUILDDIR>>/debian/build/deb/testsuite/tools G_TEST_SRCDIR=/<<PKGBUILDDIR>>/testsuite/tools TEST_OUTPUT_SUBDIR=x11 /usr/bin/bash validate

Unfortunately, the only log output for this was:

1..9
not ok 1 invalid1
not ok 2 invalid2
not ok 3 invalid3
not ok 4 invalid4
not ok 5 invalid5
not ok 6 valid1
not ok 7 valid2
not ok 8 valid3
not ok 9 valid4

I notice that many tests have this stderr output:

> MESA: error: ZINK: vkCreateInstance failed (VK_ERROR_INCOMPATIBLE_DRIVER)
> libEGL warning: egl: failed to create dri2 screen
> MESA: error: ZINK: vkCreateInstance failed (VK_ERROR_INCOMPATIBLE_DRIVER)
> glx: failed to create drisw screen
> failed to load driver: zink

Looking at the implementation of the testsuite/tools/validate test,
the way it works is: run GTK's validator against a valid or invalid
input, compare the resulting stderr with what was expected, and fail if
they differ. I think the reason why it's failing is that it's seeing this
extra stderr from Zink.

Obviously, this is quite fragile, because anything that emits a diagnostic
message can break it; but I also don't see any way for GTK upstream to get
the behaviour they want ("assert that the validator produces the messages
that we think it should") without that fragility.

Could this output to stderr in Zink perhaps be reconsidered upstream?

    smcv