[Reproducible-builds] Reproducible patches for libisoburn and libisofs

[Thomas Schmitt]

Hi,

(this time i hopefully managed to exempt you from Pkg-libburnia-devel
 list moderation)

Requests for agreement:

- Do you agree that a new -as mkisofs option

     --set_all_file_dates "$timestring"

  which sets atime, mtime, and ctime of all files, covers your patches
     0001-source_date_epoch.patch
     0002-set-default-timestamp.patch
  ?

- Do you agree that i omit
     0003-set-scdbackup_tag_time-from-source_date_epoch.patch
  and rather expect that option  --scdbackup_tag  is not to be used when
  reproducibly building an ISO ?
  (Reason is the meaning of the tag in the context of my backup tool
   scdbackup. A fake timestamp would be flatly wrong.)
  
- Do you agree with

     http://libburnia-project.org/changeset/5733

  which i derived from your
     0004-normalize-wday-yday.patch


------------------------------------------------------------------------
Details:
------------------------------------------------------------------------

i wrote:
> > Is it really desirable/correct to override the timestamps of files when
> > they get put into the ISO image ?

Chris Lamb wrote:
> At least for modification times, I think we *could* go
> with asking users to run something like:
>   $ find sourcedir/ -newermt '$SOURCE_DATE_EPOCH' -print0 |
>        xargs -0r touch --no-dereference --date='$SOURCE_DATE_EPOCH'
> .. prior to building, but AIUI one cannot reliably set atime and
> ctime so I ended up just overridding if-and-only-if SOURCE_DATE_EPOCH
> is set.

Well, if date manipulation is desired, then we can already do it in xorriso.
"RockRidge filesystem manipulator" ... yada yada ... :))

The only problem is the inconvenience to leave -as mkisofs emulation mode
and to issue two -alter_date_r commands:

  xorriso -as mkisofs ...mkisofs.options... -- \
          -alter_date_r b "$timestring" / -- \
          -alter_date_r c "$timestring" / --

A convenience option for -as mkisofs emulation could do the trick (and
make the libisoburn patches obsolete).
How about this:

  xorriso -as mkisofs mkisofs.options... --set_all_file_dates "$timestring" ...

------------------------------------------------------------------------

> > What about other file attributes which might change without
> > file content change ?

> Such as? :)  I didn't think that the ISO format really supported "much".

Rock Ridge records the attributes from stat(2): owner, group, permissions,
file type. Linux mount(8) then shows them. People often use option -r
to make all files read-only and owned by user id 0.
Device files and Unix pipes will hardly appear. But softlinks are not
uncommon.
 
------------------------------------------------------------------------

> > > +time_t Xorriso__current_time()
> > > +        return now; 

> To be honest I now feel it should probably exit(EXIT_FAILURE), otherwise
> we're just confusing users

Well, on the level of libisoburn patches this would now be a FAILURE event
if xorriso cannot decode the timestamp argument of the new option
--set_all_file_dates.
By default, FAILURE in batch mode of xorriso yields abort and non-zero exit
value.

Probably there will be no new function  iso_current_time()  either.
But if it emerges, then it cannot call exit(), but rather would put a
FAILURE event into the message queue, so that the application above libisofs
can decide what to do.

------------------------------------------------------------------------

(I see that i did not read all your info while diving into the patches.
 Organisational blindness combined with programmer's enthusiasm.)

> > Where did you get negative effects from not setting  erg->wday , erg->yday

> Only in the "Understanding timestring" log messages.

Good catch.
  xorriso : DEBUG : Understanding ECMA-119 timestring '2016080323051000' as:  Sun Aug  3 23:05:10 2016

Re-reading man 3 mktime i now understand why your patch corrects the day
name. The line

  normalized = mktime(erg);

does the trick by modifying members of erg, whereas the following line
is without lasting effect:

  erg = gmtime(&normalized);

mktime() is quite aggressive. It manipulates not only the day numbers if
it deems it necessary. So i only let it operate on a copy of erg and
rather risk to get a wrong weekday number in case of hour rollover due
to timezone or DST oddities.
(--modification-date is intended as interface to  GRUB command
 "search --fs-uuid")

See
  http://libburnia-project.org/changeset/5733

------------------------------------------------------------------------

> > > 0003-isohybrid-mbr.patch
> > This weakens the weak random number even in the case of non-constant time:

> its a terrible PRNG in either case so it can hardly matter. ;)

It shall prevent collisions. Within a day, i'd say it gives at least
24 bits of entropy. Considering the birthday paradox, this should be
good enough for about 4000 ISOs per day.

But actually nobody could tell me what this MBR id is good for.
I got it from the isohybrid script.

Now i ponder whether it is worth to make it user-adjustable or whether
i shall derive a reproducible value from the override value of
--modification-date=.

------------------------------------------------------------------------

I am still exploring the necessity to do something about  target->now
in libisofs.

------------------------------------------------------------------------

Have a nice day :)

Thomas