[Reproducible-builds] Reproducible patches for libisoburn and libisofs

[Thomas Schmitt]

Hi,

i wrote:
> > It would be quite cumbersome to produce upstream 1.4.5 releases of the
> > libraries for Sid.

Chris Lamb wrote:
> It was just a general and friendly offer, I didn't mean for it to come 
> across as a request or require a time-consuming justification if you did
> not want to proceed.

I felt the need to explain why Sid has to wait for 1.4.6 and why potential
testers don't have to wait but should use GNU xorriso-1.4.5 which is easy
to build and runs without system-wide installation.


Newly uploaded:

  http://www.gnu.org/software/xorriso/xorriso-1.4.5.tar.gz
  MD5 e89f717787749a1331e8213c0684cda0
  Version timestamp :  2016.08.12.185822

Changesets for reproducible GPT GUIDs:
  http://bazaar.launchpad.net/~libburnia-team/libisofs/scdbackup/revision/1331
  http://libburnia-project.org/changeset/5741
  http://libburnia-project.org/changeset/5742


Are there proposals where to publicly expose the advise for reproducibility
via xorriso -as mkisofs emulation as it is now ?
----------------------------------------------------------------------

- Use xorriso-1.4.5 snapshot 2016.08.12.185822 or newer.
      (Will after release become: Use xorriso-1.4.6 or newer.)

- Use option

     --modification-date=YYYYMMDDhhmmsscc

  to control the timestamps of the filesystem superblocks and other global
  components of the ISO filesystem.

- If you let xorriso produce GPT, use option

    --gpt_disk_guid modification-date

  to produce GUIDs of low quality, or obtain once a better GUID string
  (e.g. 2303cd2a-73c7-424a-a298-25632da7f446) and use it with each 
  reproducible ISO production run by
    --gpt_disk_guid 2303cd2a-73c7-424a-a298-25632da7f446

- Consider to use option
     --set_all_file_dates YYYYMMDDhhmmsscc
  to override the timestamps of the input files and directories.

- Consider to use option
     -r
  to override POSIX ownership and access permissions.

----------------------------------------------------------------------

Although grub-mkrescue probably can live with poor GPT GUIDs, i meanwhile
found a use case in xorriso where user defined modification-date does not
express the desire for reproducibile GUIDs: xorriso command
-boot_image "any" "replay".
If xorriso modifies a bootable ISO made by grub-mkrescue, then it has
to maintain the modification date so that GRUB2 after waking up finds
the ISO. It is then inappropriate to keep GPT GUIDs, because the ISOs
are nevertheless not meant to be identical.

So the default of new option --gpt_disk_guid is old behavior "random".

----------------------------------------------------------------------

What to test:

- If you produce bootable ISOs, then please do with current xorriso-1.4.5
  and try whether the resulting ISOs play as well with boot firmware and
  partition editors as do the ISOs from previous xorriso versions.

- If you are interested in reproducible ISOs, then please try whether
  above advise suffices to get identical ISOs from equivalent file trees
  on different computers, in different timezones, and on different days.
  Use as many xorrisofs options as you can find in scripts or the web.

Tested so far:

I reproducibly rebuilt my ISO image collection of 65 GB in 126 files by two
passes, one of them under valgrind supervision. The MD5s of the resulting
ISOs were recorded. No differences were found between the passes.

Among the ISOs were 35 bootable Debian ISOs, mostly version 7 or newer.
So it seems that the exotic boot sectors impose no new instability.
(I'm just not sure whether i got an ISO of every arch. And the powerpc
 ISO will not work due to lack of HFS ...)
 

Have a nice day :)

Thomas