Bug#859103: strip-nondeterminism: does not replace all timestamps in zip archives
beuc at beuc.net
beuc at beuc.net
Thu Mar 30 11:01:40 UTC 2017
Package: strip-nondeterminism
Version: 0.032-1
Hi,
It seems strip-nondeterminism replaces correctly replaces timestamps
in Unix-specific extra fields, however this may only happen in the
central directory and not in the individual members themselves.
reprotest - using a 0xAAAA timestamp (2004-09-10) for clarity:
$ reprotest 'mkdir test; touch test/a test/b; zip -r test.zip test; strip-nondeterminism -T 1094795585 test.zip; chmod 644 test.zip; cp test.zip /tmp; sleep 2' 'test.zip'
STARTING VIRTUAL SERVER ['/usr/lib/python3/dist-packages/reprotest/virt/null']
reprotest [12:29:15]: version @version@
reprotest [12:29:15]: host decision; command line: /usr/bin/reprotest 'mkdir test; touch test/a test/b; zip -r test.zip test; strip-nondeterminism -T 1094795585 test.zip; chmod 644 test.zip; cp test.zip /tmp; sleep 2' test.zip
reprotest [12:29:15]: testbed dpkg architecture: amd64
reprotest [12:29:15]: testbed running kernel: Linux 4.9.0-1-amd64 #1 SMP Debian 4.9.6-3 (2017-01-28)
will vary: environment
will vary: fileordering
will vary: home
will vary: kernel
will vary: locales
will vary: exec_path
will vary: time
will vary: timezone
will vary: umask
copying /home/me/workdir/test/ over to virtual server's /tmp/autopkgtest.L1evFW/control/
copying /home/me/workdir/test/ over to virtual server's /tmp/autopkgtest.L1evFW/experiment/
starting build with source directory: /tmp/autopkgtest.L1evFW/control/, artifact pattern: test.zip
executing: ( umask 0022 && cd /tmp/autopkgtest.L1evFW/control/ && linux64 --uname-2.6 sh -ec 'mkdir test; touch test/a test/b; zip -r test.zip test; strip-nondeterminism -T 1094795585 test.zip; chmod 644 test.zip; cp test.zip /tmp; sleep 2' )
adding: test/ (stored 0%)
adding: test/a (stored 0%)
adding: test/b (stored 0%)
starting build with source directory: /tmp/autopkgtest.L1evFW/experiment/, artifact pattern: test.zip
executing: if ( mv /tmp/autopkgtest.L1evFW/experiment/ /tmp/autopkgtest.L1evFW/experiment-before-disorderfs/ && mkdir -p /tmp/autopkgtest.L1evFW/experiment/ && sh -ec 'disorderfs --shuffle-dirents=yes --multi-user="$(if [ $(id -u) = 0 ]; then echo yes; else echo no; fi)" "$@"' - /tmp/autopkgtest.L1evFW/experiment-before-disorderfs/ /tmp/autopkgtest.L1evFW/experiment/ && umask 0002 && cd /tmp/autopkgtest.L1evFW/experiment/ && faketime +373days+7hours+13minutes linux32 sh -ec 'mkdir test; touch test/a test/b; zip -r test.zip test; strip-nondeterminism -T 1094795585 test.zip; chmod 644 test.zip; cp test.zip /tmp; sleep 2' ); then
( __c=0; fusermount -u /tmp/autopkgtest.L1evFW/experiment/ || __c=$?; rmdir /tmp/autopkgtest.L1evFW/experiment/ || __c=$?; mv /tmp/autopkgtest.L1evFW/experiment-before-disorderfs/ /tmp/autopkgtest.L1evFW/experiment/ || __c=$?; exit $__c; );
else
__x=$?;
if ( __c=0; fusermount -u /tmp/autopkgtest.L1evFW/experiment/ || __c=$?; rmdir /tmp/autopkgtest.L1evFW/experiment/ || __c=$?; mv /tmp/autopkgtest.L1evFW/experiment-before-disorderfs/ /tmp/autopkgtest.L1evFW/experiment/ || __c=$?; exit $__c; ); then exit $__x; else
echo >&2; "cleanup failed with exit code $?"; exit $__x;
fi;
fi
disorderfs: shuffling dirents
disorderfs: reversing dirents
adding: test/ (stored 0%)
adding: test/b (stored 0%)
adding: test/a (stored 0%)
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = "fr_CH.UTF-8:fr",
LC_ALL = "fr_CH.UTF-8",
LANG = "fr_CH.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
copying /tmp/autopkgtest.L1evFW/control-dist/ back from virtual server's /tmp/tmpnnnyuzyg/control_artifact/
copying /tmp/autopkgtest.L1evFW/experiment-dist/ back from virtual server's /tmp/tmpnnnyuzyg/experiment_artifact/
Running diffoscope: ['diffoscope', '/tmp/tmpnnnyuzyg/control_artifact/', '/tmp/tmpnnnyuzyg/experiment_artifact/']
--- /tmp/tmpnnnyuzyg/control_artifact/
+++ /tmp/tmpnnnyuzyg/experiment_artifact/
│ --- /tmp/tmpnnnyuzyg/control_artifact/test.zip
├── +++ /tmp/tmpnnnyuzyg/experiment_artifact/test.zip
│┄ No file format specific differences found inside, yet data differs (Zip archive data, at least v1.0 to extract)
│ @@ -1,18 +1,18 @@
│ 00000000: 504b 0304 0a00 0000 0000 a22e 2a31 0000 PK..........*1..
│ 00000010: 0000 0000 0000 0000 0000 0500 1c00 7465 ..............te
│ 00000020: 7374 2f55 5409 0003 4141 4141 4141 4141 st/UT...AAAAAAAA
│ 00000030: 7578 0b00 0104 0000 0000 0400 0000 0050 ux.............P
│ 00000040: 4b03 040a 0000 0000 00a2 2e2a 3100 0000 K..........*1...
│ 00000050: 0000 0000 0000 0000 0006 001c 0074 6573 .............tes
│ -00000060: 742f 6155 5409 0003 fbdd dc58 fbdd dc58 t/aUT......X...X
│ +00000060: 742f 6155 5409 0003 fddd dc58 fddd dc58 t/aUT......X...X
│ 00000070: 7578 0b00 0104 e803 0000 04e8 0300 0050 ux.............P
│ 00000080: 4b03 040a 0000 0000 00a2 2e2a 3100 0000 K..........*1...
│ 00000090: 0000 0000 0000 0000 0006 001c 0074 6573 .............tes
│ -000000a0: 742f 6255 5409 0003 fbdd dc58 fbdd dc58 t/bUT......X...X
│ +000000a0: 742f 6255 5409 0003 fddd dc58 fddd dc58 t/bUT......X...X
│ 000000b0: 7578 0b00 0104 e803 0000 04e8 0300 0050 ux.............P
│ 000000c0: 4b01 021e 030a 0000 0000 00a2 2e2a 3100 K............*1.
│ 000000d0: 0000 0000 0000 0000 0000 0005 0018 0000 ................
│ 000000e0: 0000 0000 0010 00ed 4100 0000 0074 6573 ........A....tes
│ 000000f0: 742f 5554 0500 0341 4141 4175 780b 0001 t/UT...AAAAux...
│ 00000100: 0400 0000 0004 0000 0000 504b 0102 1e03 ..........PK....
│ 00000110: 0a00 0000 0000 a22e 2a31 0000 0000 0000 ........*1......
5 ouf of 9 timestamps were replaced, 4 of them are remaining as
confirmed by bsdtar:
$ bsdtar -tvf /tmp/test.zip
drwxr-xr-x 0 0 0 0 sept. 10 2004 test/
-rw-r--r-- 0 1000 1000 0 mars 30 12:29 test/a
-rw-r--r-- 0 1000 1000 0 mars 30 12:29 test/b
I couldn't precisely locate where the error is in the
File/StripNondeterminism/handlers/zip.pm - maybe removing and adding
back a zip member only resets the directory extra fields.
Cheers!
Sylvain
More information about the Reproducible-builds
mailing list