Bug#966923: meson: 0.55.0 considers TAP "ok # SKIP" to be a failure

[Simon McVittie]

Control: retitle -1 meson: 0.55.0 considers TAP "ok # SKIP" to be a failure
Control: reassign -1 meson 0.55.0-2
Control: affects -1 + gjs
Control: forwarded -1 https://github.com/mesonbuild/meson/issues/7515
Control: tags -1 + upstream

On Mon, 03 Aug 2020 at 10:51:25 +0200, Lucas Nussbaum wrote:
> Relevant part (hopefully):
...
> > meson test -C /<<PKGBUILDDIR>>/obj-x86_64-linux-gnu --timeout-multiplier 4
...
> > 10/53 gjs:JS / GIMarshalling            FAIL           0.22s (exit status 0)
...
> > 27/53 gjs:JS / Regress                  FAIL           0.17s (exit status 0)
...
> > 33/53 gjs:JS / Gtk3                     FAIL           0.22s (exit status 0)

What is happening here was obscured by gjs not printing the output of
failing tests to the buildd log, but if you apply the attached gjs.diff,
you can see that the affected tests are those that have some steps that
are skipped.

Reproducer:

$ cat meson.build
project('debian-bug-966923')
foreach test_name : ['pass.sh', 'skip.sh']
  test(test_name, files(test_name), protocol : 'tap')
endforeach
$ cat pass.sh
#!/bin/sh
echo "ok 1"
echo "ok 2"
exit 0
$ cat skip.sh
#!/bin/sh
echo "ok 1 # SKIP cannot actually do this"
echo "ok 2"
exit 0
$ chmod +x pass.sh skip.sh
$ meson _build
$ meson test -C _build

Regards,
    smcv
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gjs.diff
Type: text/x-diff
Size: 48 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-gnome-maintainers/attachments/20200803/c76422a5/attachment.diff>