[Reproducible-builds] generating reproducible ISOs with xorriso

[Thomas Schmitt]

Hi,

> I participate in the Debian Reproducible Builds project

I saw some sin-list page about packages shortly after
xorriso-1.4.0 was released.
It complained about libburn's doc/doxygen.conf.in which i hope to
have fixed by
  http://libburnia-project.org/changeset/5446


> I wanted to see if xorriso would be
> interested in offering a "reproducible" option during ISO creation.
> [...]
> The variation within an ISO can come from many places, probably
> including:
> * filesystem timestamps

Afaik, GRUB2 uses the ISO 9660 Modification Date as UUID of the ISO.
There are several ISO 9660 timestamps. One would have to fixate
them by explicit xorriso commands.


>  * extent ordering/numbering (maybe derived from source filesystem
>  ordering)

The dull ISO 9660 names get sorted and their data content
is stored in the sequence of that sorting. So if the tree of
paths stays unaltered, the sequence of data extents should
stay unaltered too.
The mapping from Rock Ridge names to ISO 9660 names is supposed
to be deterministic in this case. (It is complicated, though.)


>  * bootable metadata (Boot offsets?  i don't know the jargon, but there
>   is a value reported by "isoinfo -d" called "Bootoff")

The second number after "Bootoff" is the block address of the
data extent of an El torito boot image data file. Thus covered
by the sorting if all file sizes are unaltered.

I riddle what the first number might be.
debian-7.7.0-amd64-netinst.iso has "Bootoff 350 848".
The first El Torito boot image begins at block 848 decimal.
Maybe just a disguised hex ? 0x350 = 848 
 

> it seems like it would be simpler if there were a way to
> tell xorriso to just make an identical image with all metadata
> standardized in some way.
> This mode might imply:
> 
>  * supplying a timestamp to be used for all imported files (like
> alter_date_r ?)

Well possible.
After the usual xorriso -as mkisofs options of debian-cd
one could add native xorriso commands. Use a timestamp
format that is not prone to time zone computation. E.g.
the native format of ISO 9660:

  timestamp="2015060415363300" 
  xorriso -as mkisofs ...all.the.debian-cd.options... \
  -- \
  -volume_date "c" "$timestamp" \
  -volume_date "m" "$timestamp" \
  -volume_date "x" "default" \
  -volume_date "f" "default" \
  -alter_date "b" "$timestamp" / -- \
  -alter_date "c" "$timestamp" / -- \

Filesystem times "x" = Expire and "f" = Valid Since are best set
to the default value 000...000. Just in case anybody cares to
obey them.


>  * sorting files included so that extent numbering is constant

As said, this is supposed to happen always. (Modulo bugs which
you might find.)


>  * ... other things?

One will have to make experiments with the ISO production
scripts which shall be used. Identify the deviating spots
in the ISO, their meaning, and their origin.

I am willing to help with advise and/or necessary new features.


Have a nice day :)

Thomas