Bug#1152143: sparse tar archives cannot be made bit-by-bit reproducible as they embed the (semi-random) process id

Johannes Schauer Marin Rodrigues josch at debian.org
Mon Oct 26 06:23:53 GMT 2026 

Package: tar
Version: 1.35+dfsg-6
Severity: normal
Tags: patch
X-Debbugs-Cc: reproducible-bugs at lists.alioth.debian.org
User: reproducible-builds at lists.alioth.debian.org
Usertags: randomness toolchain
Control: forwarded -1 https://lists.gnu.org/archive/html/bug-tar/2026-10/msg00009.html

[The bug is fixed upstream. This is a tracking bug in Debian for this issue]

Hi,

I want to create bit-by-bit reproducible sparse tar archives of the same sparse
input file and find myself unable to do so using gnu tar. I create the sparse
test input like this:

    $ echo start > sparsefile
    $ truncate -s 4M sparsefile
    $ echo middle >> sparsefile
    $ truncate -s 8388604 sparsefile
    $ echo end >> sparsefile
    $ stat -c '%b*%B-%s' -- sparsefile
    24*512-8388608
    $ sha1sum sparsefile
    e4d6196039c65beb8ba41b5b45f404549e41303c  sparsefile

The output of stat shows that indeed the file I created contains holes as the
number of allocated blocks (24) is much smaller than the size of the file
(8MiB) when multiplied by the block size (512).

I try to pack up the file `sparsefile` in a tarball using this invocation of
gnu tar:

    $ SOURCE_DATE_EPOCH=0 tar --mtime=@0 --pax-option='delete=atime,delete=ctime' --numeric-owner --owner=0 --group=0 --format=pax --sparse -cf sparse-archive1.tar sparsefile

I then repeat the process, only changing the output filename:

    $ SOURCE_DATE_EPOCH=0 tar --mtime=@0 --pax-option='delete=atime,delete=ctime' --numeric-owner --owner=0 --group=0 --format=pax --sparse -cf sparse-archive2.tar sparsefile

My expectation is, that `sparse-archive1.tar` and `sparse-archive2.tar` are
identical as they contain the same file, I have $SOURCE_DATE_EPOCH set, I
delete atime and ctime from the pax header, I set the mtime to 0 and force the
owner to be root. But despite all of that, `sparse-archive1.tar` and
`sparse-archive2.tar` differ like this when looking at them using xxd:

@@ -59,23 +59,23 @@
 000003a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000003b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000003c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000003d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000003e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000003f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000400: 2e2f 474e 5553 7061 7273 6546 696c 652e  ./GNUSparseFile.
-00000410: 3138 3636 3433 372f 7370 6172 7365 6669  1866437/sparsefi
+00000410: 3138 3636 3432 342f 7370 6172 7365 6669  1866424/sparsefi
 00000420: 6c65 0000 0000 0000 0000 0000 0000 0000  le..............
 00000430: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000440: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000450: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000460: 0000 0000 3030 3030 3636 3400 3030 3030  ....0000664.0000
 00000470: 3030 3000 3030 3030 3030 3000 3030 3030  000.0000000.0000
 00000480: 3030 3331 3030 3000 3030 3030 3030 3030  0031000.00000000
-00000490: 3030 3000 3031 3334 3236 0020 3000 0000  000.013426. 0...
+00000490: 3030 3000 3031 3334 3232 0020 3000 0000  000.013422. 0...
 000004a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000004b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000004c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000004d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000004e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000004f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000500: 0075 7374 6172 0030 3000 0000 0000 0000  .ustar.00.......


The checksum differs because the stored file contains the tar pid 1866437 for
`sparse-archive1.tar` and 1866424 for `sparse-archive2.tar`.

I read https://www.gnu.org/software/tar/manual/html_node/PAX-0.html to find out
that the sparse file is stored using the pattern %d/GNUSparseFile.%p/%f (where
%p is the PID) so that tar implementations that are not aware of gnu extensions
store the sparse file into separate directories.

Workarounds I found so far:

 1. unshare the pid namespace
-----------------------------

If I run tar prefixed with

    $ unshare --map-auto --map-root-user --fork --pid -- tar ...

Then tar will run inside a new pid namespace and will always run with a pid of
1 in that namespace. To create a new pid namespace without being root, I also
need to unshare the user namespace, so I am likely no longer allowed to write
into the current directory as the unshared user. To work around that I can
let tar write the tarball to stdout and then redirect that into a file. Full
commandline:

    $ unshare --map-auto --map-root-user --fork --pid -- tar --clamp-mtime 
--mtime=@0 --pax-option='delete=atime,delete=ctime' --numeric-owner --owner=0 
--group=0 --format=pax --sparse -cf - sparsefile > sparse-archive.tar

 2. use PAX format 0.0
----------------------

The initial format does not embed the PID into the tarball, so I get bit-by-bit
reproducible output if I run tar like this:

    $ tar --sparse-version 0.1 --pax-option delete=GNU.sparse.map --clamp-mtime 
--mtime=@0 --pax-option='delete=atime,delete=ctime' --numeric-owner --owner=0 
--group=0 --            format=pax --sparse -cf sparse-archive.tar sparsefile

Using format 0.0 is probably not recommended for reasons explained in the
documentation I linked above.

 3. don't use gnu tar
---------------------

bsdtar from libarchive does not include the PID into the output and uses a
static value of 0 instead:

    $ bsdtar --clamp-mtime --mtime=@0 --numeric-owner --uid 0 --gid 0 
--read-sparse -cf sparse-archive.tar sparsefile

But this is what the bsdtar man page calls "restricted pax format". If I run
bsdtar with --format pax then the resulting tarball becomes unreproducible
again because in contrast to gnu tar, bsdtar does not have a way to remove the
atime and ctime pax options from the header. Even when setting the atime using
the touch utility, I cannot control the ctime.

In summary, by default gnu tar is unable to produce bit-by-bit identical sparse
tarballs from the same sparse input file.

I attached a patch from upstream (minus the documentation changes) which fixes
the issue.

Thanks!

cheers, josch
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-tar-make-GNUSparseFile-headers-reproducible.patch
Type: application/mbox
Size: 2462 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/reproducible-bugs/attachments/20261026/93c4a4bb/attachment.mbox>

More information about the Reproducible-bugs mailing list