[Reproducible-builds] [xz-devel] Re: xz-utils: make the selected POSIX shell stable accross build environments

[Ximin Luo]

+bugs-gnulib, reproducible-builds

Lasse Collin:
> On 2016-06-07 Ximin Luo wrote:
>> I've attached a patch that makes m4/posix-shell.m4 try constant paths
>> first. This should fix the issue.
>>
>> Upstream should also apply it - see more-stable-shell.patch.
> 
> Thanks!
> 
> posix-shell.m4 comes from gnulib so it would be nice if you could send
> the patch there and discuss the issue with gnulib developers. Perhaps
> there is a reason why the shells are tested in that order, although the
> gnulib commit messages don't have any reasoning. A guess is that
> someone might prefer if the same shell is used for running
> configure and the test suite scripts. Anyway, getting it fixed in gnulib
> would get it fixed in projects other than XZ Utils too.
> 

Thanks, yes this wasn't clear to me. We'd still need to contact projects that have already copied posix-shell.m4 into their source tree, but I suppose at least future projects will benefit.

bugs-gnulib, do you see any issue with this patch? The context is that some projects embed POSIX_SHELL into build products, so for build reproducibility it is better to have this detection script first try constant paths.

(Yes, if /bin/sh is not POSIX for one of the build machines, then we still have unreproducibility. But that's much less likely to happen. This is an "effort vs correctness" trade-off that I'm making.)

Regarding "someone might prefer if the same shell" as Lasse theorised, I'm not sure this will ever be an issue: if they prefer "the same shell", they would just use $SHELL or $CONFIG_SHELL instead of $POSIX_SHELL, so the only useful scenario is if somehow they wanted a shell that was POSIX-compatible but for some reason (I can't imagine why) needed this selection to favour $SHELL/$CONFIG_SHELL but still for it to be OK to fall back to the other constant values.

> One can force the POSIX shell to a specific value on the configure
> command line by passing, for example, "gl_cv_posix_shell=/bin/sh" as an
> argument. It's not documented in the --help output but it's mentioned
> in INSTALL section 3.1. That is an alternative to patching to get
> reproducible builds.
> 

Yeah, I saw that too, but thought this approach was a bit cleaner.

X

-- 
GPG: ed25519/56034877E1F87C35
GPG: rsa4096/1318EFAC5FBBDBCE
git://github.com/infinity0/pubkeys.git
-------------- next part --------------
Description: More stable shell in build output
 When detecting a POSIX shell, try constant systems paths first because that is
 more likely to give a reproducible build output.
Author: Ximin Luo <infinity0 at debian.org>
Bug: https://www.mail-archive.com/xz-devel@tukaani.org/
Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=806331
---
This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
--- a/m4/posix-shell.m4
+++ b/m4/posix-shell.m4
@@ -43,7 +43,7 @@
        subshell_umask_sanity
      '
      for gl_cv_posix_shell in \
-	 "$CONFIG_SHELL" "$SHELL" /bin/sh /bin/bash /bin/ksh /bin/sh5 no; do
+	 /bin/sh /bin/bash /bin/ksh /bin/sh5 "$CONFIG_SHELL" "$SHELL" no; do
        case $gl_cv_posix_shell in
          /*)
 	   "$gl_cv_posix_shell" -c "$gl_test_posix_shell_script" 2>/dev/null \