[Pkg-zsh-devel] Bug#935115: bash: [regression] passing variable assignments to functions broken in POSIX mode, violating POSIX

[Thorsten Glaser]

Package: bash
Version: 5.0-4
Severity: critical
Justification: breaks unrelated software

Found this gem in #934027:

tglase at tglase:~ $ cat testscript
dbc_mysql_createdb(){
	local ret l_dbname _dbc_nodb

	echo "dbc_mysql_createdb: _dbc_nodb(1)=$_dbc_nodb"
	_dbc_nodb="yes" dbc_mysql_exec_command "CREATE DATABASE \`$dbc_dbname\`${extrasql:-}"
	ret=$?
	echo "dbc_mysql_createdb: _dbc_nodb(2)=$_dbc_nodb"	# POSIX: unspecified
	/usr/bin/env | fgrep -c _dbc_nodb | sed 's/^/after: /'	# POSIX: unspecified
	echo dbc_mysql_createdb:$ret
}

dbc_mysql_exec_command(){
	local statement l_sqlfile l_retval
	statement="$@"
	l_retval=0
	echo "dbc_mysql_exec_command: _dbc_nodb=$_dbc_nodb"	# POSIX: required
	/usr/bin/env | fgrep -c _dbc_nodb | sed 's/^/inner: /'	# POSIX: unspecified
	return $l_retval
}

dbc_mysql_createdb
tglase at tglase:~ $ dash testscript
dbc_mysql_createdb: _dbc_nodb(1)=
dbc_mysql_exec_command: _dbc_nodb=yes
inner: 0
dbc_mysql_createdb: _dbc_nodb(2)=yes
after: 0
dbc_mysql_createdb:0
tglase at tglase:~ $ ksh -c 'alias local=typeset; . ./testscript'
dbc_mysql_createdb: _dbc_nodb(1)=
dbc_mysql_exec_command: _dbc_nodb=yes
inner: 0
dbc_mysql_createdb: _dbc_nodb(2)=yes
after: 0
dbc_mysql_createdb:0
tglase at tglase:~ $ lksh -o posix testscript  # native mksh +o posix is identical
dbc_mysql_createdb: _dbc_nodb(1)=
dbc_mysql_exec_command: _dbc_nodb=yes
inner: 1
dbc_mysql_createdb: _dbc_nodb(2)=yes
after: 1
dbc_mysql_createdb:0
tglase at tglase:~ $ posh testscript
dbc_mysql_createdb: _dbc_nodb(1)=
dbc_mysql_exec_command: _dbc_nodb=yes
inner: 1
dbc_mysql_createdb: _dbc_nodb(2)=
after: 0
dbc_mysql_createdb:0
tglase at tglase:~ $ yash testscript
dbc_mysql_createdb: _dbc_nodb(1)=
dbc_mysql_exec_command: _dbc_nodb=yes
inner: 1
dbc_mysql_createdb: _dbc_nodb(2)=
after: 0
dbc_mysql_createdb:0
tglase at tglase:~ $ zsh -c 'emulate sh; . ./testscript'
dbc_mysql_createdb: _dbc_nodb(1)=
dbc_mysql_exec_command: _dbc_nodb=yes
inner: 1
dbc_mysql_createdb: _dbc_nodb(2)=yes
after: 1
dbc_mysql_createdb:0
tglase at tglase:~ $ bash --version 2>&1 | head -1
GNU bash, version 5.0.3(1)-release (x86_64-pc-linux-gnux32)
tglase at tglase:~ $ bash testscript
dbc_mysql_createdb: _dbc_nodb(1)=
dbc_mysql_exec_command: _dbc_nodb=yes
inner: 1
dbc_mysql_createdb: _dbc_nodb(2)=
after: 0
dbc_mysql_createdb:0
tglase at tglase:~ $ bash --posix testscript
dbc_mysql_createdb: _dbc_nodb(1)=
dbc_mysql_exec_command: _dbc_nodb=
inner: 1
dbc_mysql_createdb: _dbc_nodb(2)=
after: 1
dbc_mysql_createdb:0
tglase at tglase:~ $ schroot -prc stretch
(stretch-i386)tglase at tglase:~ $ bash --version 2>&1 | head -1
GNU bash, version 4.4.12(1)-release (i686-pc-linux-gnu)
(stretch-i386)tglase at tglase:~ $ bash testscript
dbc_mysql_createdb: _dbc_nodb(1)=
dbc_mysql_exec_command: _dbc_nodb=yes
inner: 1
dbc_mysql_createdb: _dbc_nodb(2)=
after: 0
dbc_mysql_createdb:0
(stretch-i386)tglase at tglase:~ $ bash --posix testscript
dbc_mysql_createdb: _dbc_nodb(1)=
dbc_mysql_exec_command: _dbc_nodb=yes
inner: 1
dbc_mysql_createdb: _dbc_nodb(2)=yes
after: 1
dbc_mysql_createdb:0


The expected output is:

dbc_mysql_createdb: _dbc_nodb(1)=	# initially not set / empty
dbc_mysql_exec_command: _dbc_nodb=yes	# MUST be visible inside the function
inner: 0 or 1				# MAY be exported, does not need to
dbc_mysql_createdb: _dbc_nodb(2)=[yes]	# MAY be visible afterwards, optional
after: 0 or 1				# if visible afterwards MAY be exported
dbc_mysql_createdb:0

POSIX reference:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_01
specifically:

 * If the command name is a function that is not a standard utility
   implemented as a function, variable assignments shall affect the
   current execution environment during the execution of the function.
   It is unspecified:
   + Whether or not the variable assignments persist after the
     completion of the function
   + Whether or not the variables gain the export attribute during the
     execution of the function
   + Whether or not export attributes gained as a result of the variable
     assignments persist after the completion of the function (if
     variable assignments persist after the completion of the function)

Tested shells:

┌──────────────┬────────────────┬──────────┬────────────────────┬──────────┐
│ shell \ what │ inside visible │ exported │ afterwards visible │ exported │
├──────────────┼────────────────┼──────────┼────────────────────┼──────────┤
│ bash4        │ ✔ as required  │ ✓ (good) │ ✗ no, permitted    │ –        │
│ bash4 posix  │ ✔ as required  │ ✓ (good) │ ✓ yes, permitted   │ ✓ (ok)   │
│ bash5        │ ✔ as required  │ ✓ (good) │ ✗ no, permitted    │ –        │
│ bash5 posix  │ ✘  !!FAIL!! ⚠  │ ✓ (huh?) │ ✗ empty            │ ✓ (huh?) │
│ dash         │ ✔ as required  │ ✗ (ok)   │ ✗ no, permitted    │ –        │
│ ksh93        │ ✔ as required  │ ✗ (ok)   │ ✗ no, permitted    │ –        │
│ mksh ±posix  │ ✔ as required  │ ✓ (good) │ ✓ yes, permitted   │ ✓ (ok)   │
│ posh (old)   │ ✔ as required  │ ✓ (good) │ ✗ no, permitted    │ –        │
│ yash         │ ✔ as required  │ ✓ (good) │ ✗ no, permitted    │ –        │
│ zsh as sh    │ ✔ as required  │ ✓ (good) │ ✓ yes, permitted   │ ✓ (ok)   │
└──────────────┴────────────────┴──────────┴────────────────────┴──────────┘

Out of all tested shells, only bash5 in POSIX mode is completely broken:
it makes an empty, but exported (huh?) variable available to both the
function’s body and afterwards but does not pass on the content.

Out of all nonbroken shells, all shells that do make the variable
visible afterwards export it, both inside and afterwards, which is
consistent: bash4 in POSIX mode, mksh in native and POSIX modes,
zsh in sh mode.

Out of all nonbroken shells, only dash and AT&T ksh93 do not export
the variable during execution of the function body, which, while
perhaps not expected by the programmer, is valid POSIX.

The “afterwards visible” property is dividing. Hiding it would be
consistent related to how assignments are handled running non-functions,
which is what the majority does (bash4/5 in nōn-POSIX mode, dash,
AT&T ksh93, posh and yash), only a very interesting minority, namely
bash4 in POSIX mode, mksh and zsh in “sh” (POSIX) mode, keep the variable
alive. (With mksh upstream hat on, and looking at pdksh, antecessor of
both posh and mksh: pdksh keeps it visible (so, this is a deliberate(?)
change in posh) but did not export it, which apparently got fixed later
in both shells.)

-- System Information:
Debian Release: bullseye/sid
  APT prefers unreleased
  APT policy: (500, 'unreleased'), (500, 'buildd-unstable'), (500, 'unstable'), (100, 'experimental')
Architecture: x32 (x86_64)
Foreign Architectures: i386, amd64

Kernel: Linux 4.19.0-5-amd64 (SMP w/4 CPU cores)
Kernel taint flags: TAINT_FIRMWARE_WORKAROUND
Locale: LANG=C, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=C (charmap=UTF-8)
Shell: /bin/sh linked to /bin/lksh
Init: sysvinit (via /sbin/init)

Versions of packages bash depends on:
ii  base-files   11
ii  debianutils  4.8.6.3
ii  libc6        2.28-10
ii  libtinfo6    6.1+20190803-1

Versions of packages bash recommends:
pn  bash-completion  <none>

Versions of packages bash suggests:
pn  bash-doc  <none>

-- no debconf information