[Pkg-zsh-devel] zsh test suite failures under reprotest on Salsa

[Axel Beckert]

Hi Daniel,

thanks a lot for looking into this.

Daniel Shahaf wrote:
> > 6585 Was testing: unreadable directories can be globbed (users/24619, users/24626)
> > 6586 ../../Test/D02glob.ztst: test failed.
> > 
> > My gut feeling currently thinks that the first issue is likely related
> > to how Salsa CI works.
> 
> This test checks unreadable/unexecutable directories are included in
> glob expansions.  It doesn't try to read the contents of those directories.
> 
> Which is to say, this isn't the run-of-the-mill scenario where a test
> creates a mode-000 directory, expects to fail to read it, and XPASSes
> when run with superuser privileges on some systems.  Rather, this test
> point tests a bug in zsh that only manifests when privilege restrictions
> _are_ in effect.

Ok. Yay, I found a bug! ;-)

> The test passes only on Salsa and fails as expected everywhere else
> (that's what the output says, though it's not immediately clear),

Yes, I was aware of this. I don't care if the test needs a "not" in
front or not. :-)

> it is plausible that the reason for the difference has to do with
> how Salsa runs the tests.

Ack.

> The bug which the test tests for was fixed upstream in workers/45291.

This seems to have been short before the 5.8 release and is hence
included, right?

> > 7576 Was testing: PRIVILEGED automatically enabled when RUID != EUID
> > 7577 ../../Test/P01privileged.ztst: test failed.
> > 
> > And for the second issue my gut feeling says this is either due to how
> > Salsa CI works or that Zsh should have detected this situation and not
> > run that test.
> 
> The test fails because the test harness was invoked with an invalid
> LD_PRELOAD value.  zsh's build system did not set up the libeatmydata.so
> preloading, so I don't see why it should have skipped the test.
> 
> Questions:
> 
> - What set up eatmydata?

Likely the container in which the test was run, i.e. either Salsa's CI
itself or the pipeline defined by the Salsa CI Team.

> - Why did the test fail with that error?

Good question.

> - Why did other *.ztst files _not_ fail?

Even better question!

Daniel Shahaf wrote:
> > > 6585 Was testing: unreadable directories can be globbed (users/24619, users/24626)
> > > 6586 ../../Test/D02glob.ztst: test failed.
> > 
> > (that's what the output says, though it's not immediately clear),
> 
> Patch sent upstream: workers/46102.

Hmmm, I don't get that patch. But maybe because understanding the
output was not my issue. :-)

		Regards, Axel
-- 
 ,''`.  |  Axel Beckert <abe at debian.org>, https://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-    |  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE