[Pkg-zsh-devel] zsh test suite failures under reprotest on Salsa

[Axel Beckert]

Hi,

today I added a debian/salsa-ci.yml file to our zsh git repo to be
able to use the generci Gitlab CI pipeline the Debian Salsa CI Team
maintains[1]. (I currently see this as an addition to Grml's Jenkins
at https://jenkins.grml.org/view/Debian/, but we might decide to only
continue to use one of them in the future.)

reprotest[2] builds the package under different environment (languages,
timezones, etc.) and checks if the resulting packages are identical.

Currently two tests fails[3] during the package build as follows:

6583 Running test: unreadable directories can be globbed (users/24619, users/24626)
6584 Test ../../Test/D02glob.ztst failed: test was expected to fail, but passed.
6585 Was testing: unreadable directories can be globbed (users/24619, users/24626)
6586 ../../Test/D02glob.ztst: test failed.

and

7563 ../../Test/P01privileged.ztst: starting.
7564 Selecting unprivileged UID:EUID pair automatically
7565 Selecting unprivileged GID:EGID pair automatically
7566 Using unprivileged UID 1, EUID 2, GID 1, EGID 2
7567 Running test: PRIVILEGED automatically enabled when RUID != EUID
7568 --- /tmp/zsh.ztst.14839/ztst.err 2020-06-21 13:45:57.442336033 -1200
7569 +++ /tmp/zsh.ztst.14839/ztst.terr 2020-06-21 13:45:57.454336024 -1200
7570 @@ -0,0 +1 @@
7571 +ERROR: ld.so: object 'libeatmydata.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
7572 Test ../../Test/P01privileged.ztst failed: error output differs from expected as shown above for:
7573   re_zsh $ruid $ruid -1 -1 'echo $UID/$EUID $options[privileged]'
7574   re_zsh $euid $euid -1 -1 'echo $UID/$EUID $options[privileged]'
7575   re_zsh $ruid $euid -1 -1 'echo $UID/$EUID $options[privileged]'
7576 Was testing: PRIVILEGED automatically enabled when RUID != EUID
7577 ../../Test/P01privileged.ztst: test failed.

Both seem to be permissions/privileges related, so I wonder what your
gut feeling says:

* Are these issues with how the Salsa CI works?
* Are these issues that Zsh's test suite doesn't properly to detect if
  an environment allows specific tests to be made.
* Are these real bugs in Zsh?

My gut feeling currently thinks that the first issue is likely related
to how Salsa CI works.

And for the second issue my gut feeling says this is either due to how
Salsa CI works or that Zsh should have detected this situation and not
run that test.

Haven't digged much deeper yet, though.

[1] https://salsa.debian.org/salsa-ci-team/pipeline/
[2] https://packages.debian.org/unstable/reprotest
[3] https://salsa.debian.org/debian/zsh/-/jobs/818139

		Regards, Axel
-- 
 ,''`.  |  Axel Beckert <abe at debian.org>, https://people.debian.org/~abe/
: :' :  |  Debian Developer, ftp.ch.debian.org Admin
`. `'   |  4096R: 2517 B724 C5F6 CA99 5329  6E61 2FF9 CD59 6126 16B5
  `-    |  1024D: F067 EA27 26B9 C3FC 1486  202E C09E 1D89 9593 0EDE