[Pkg-zsh-devel] zsh test suite failures under reprotest on Salsa

[Daniel Shahaf]

Axel Beckert wrote on Mon, 22 Jun 2020 04:24 +0200:
> * Are these issues with how the Salsa CI works?
> * Are these issues that Zsh's test suite doesn't properly to detect if
>   an environment allows specific tests to be made.
> * Are these real bugs in Zsh?

> 6583 Running test: unreadable directories can be globbed (users/24619, users/24626)
> 6584 Test ../../Test/D02glob.ztst failed: test was expected to fail, but passed.
> 6585 Was testing: unreadable directories can be globbed (users/24619, users/24626)
> 6586 ../../Test/D02glob.ztst: test failed.
> 
> My gut feeling currently thinks that the first issue is likely related
> to how Salsa CI works.

This test checks unreadable/unexecutable directories are included in
glob expansions.  It doesn't try to read the contents of those directories.

Which is to say, this isn't the run-of-the-mill scenario where a test
creates a mode-000 directory, expects to fail to read it, and XPASSes
when run with superuser privileges on some systems.  Rather, this test
point tests a bug in zsh that only manifests when privilege restrictions
_are_ in effect.

The test passes only on Salsa and fails as expected everywhere else
(that's what the output says, though it's not immediately clear), it is
plausible that the reason for the difference has to do with how Salsa
runs the tests.

The bug which the test tests for was fixed upstream in workers/45291.

> 7563 ../../Test/P01privileged.ztst: starting.
> 7564 Selecting unprivileged UID:EUID pair automatically
> 7565 Selecting unprivileged GID:EGID pair automatically
> 7566 Using unprivileged UID 1, EUID 2, GID 1, EGID 2
> 7567 Running test: PRIVILEGED automatically enabled when RUID != EUID
> 7568 --- /tmp/zsh.ztst.14839/ztst.err 2020-06-21 13:45:57.442336033 -1200
> 7569 +++ /tmp/zsh.ztst.14839/ztst.terr 2020-06-21 13:45:57.454336024 -1200
> 7570 @@ -0,0 +1 @@
> 7571 +ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
> 7572 Test ../../Test/P01privileged.ztst failed: error output differs from expected as shown above for:
> 7573   re_zsh $ruid $ruid -1 -1 'echo $UID/$EUID $options[privileged]'
> 7574   re_zsh $euid $euid -1 -1 'echo $UID/$EUID $options[privileged]'
> 7575   re_zsh $ruid $euid -1 -1 'echo $UID/$EUID $options[privileged]'
> 7576 Was testing: PRIVILEGED automatically enabled when RUID != EUID
> 7577 ../../Test/P01privileged.ztst: test failed.
> 
> And for the second issue my gut feeling says this is either due to how
> Salsa CI works or that Zsh should have detected this situation and not
> run that test.

The test fails because the test harness was invoked with an invalid
LD_PRELOAD value.  zsh's build system did not set up the libeatmydata.so
preloading, so I don't see why it should have skipped the test.

Questions:

- What set up eatmydata?

- Why did the test fail with that error?

- Why did other *.ztst files _not_ fail?

Cheers,

Daniel