Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files

[Sebastiaan Couwenberg]

On 5/8/23 22:43, Vagrant Cascadian wrote:
> On 2023-05-08, Sebastiaan Couwenberg wrote:
>> On 5/8/23 02:07, Vagrant Cascadian wrote:
>>> The attached patch fixes this by not touching these files during the
>>> build process.
>>
>>   From shar(1):
>>
>> "
>>     -m, --no-timestamp
>>            do not restore modification times.
>>
>>            Avoid generating 'touch' commands to restore the file
>>            modification dates when unpacking files from the archive.
>>
>>            When file modification times are not preserved, project build
>>            programs like "make" will see built files older than the files
>>            they get built from.  This is why, when this option is not
>>            used, a special effort is made to restore timestamps.
>> "
>>
>> That should be used when generating the archives instead of your patch
>> to not have the mtimes touched when unpacking the archives.
> 
> Is it actually a problem to allow dpkg to normalize the timestamps on
> these files rather than forcefully setting them to ... a value from a
> shar archive? It is perhaps a naive question; I really do not know.

Where does dpkg normalize the timestamps?

shar sets the timestamps when the archive is unpacked before the package 
built starts.

Some of the files in the diffoscope-results are only installed in 
proj-data and not used otherwise during the build.

  * BETA2007.gsb is used in test/gie/DHDN_ETRS89.gie

  * CHENYX06.gsb/CHENYX06_etrs.gsb/CHENYX06a.gsb are only installed

  * egm96_15.gtx is used in test/gie/deformation.gie,
    test/gie/more_builtins.gie, test/gie/4D-API_cs2cs-style.gie, and
    test/cli/testdatumfile

  * ntf_r93.gsb is used in test/gie/more_builtins.gie,
    test/gie/4D-API_cs2cs-style.gie, and test/cli/testdatumfile

  * nzgd2kgrid0005.gsb is used in unit tests

>> But the diffoscope-results only show a few of the files from the shar
>> archives with different mtimes, and all of them get touched when
>> unpacking the archive just before the configure target is executed.
> 
> Well, I too was perplexed why other files were not affected, but it is
> consistently those .gsb and .gtx files, and the submitted patch allows
> to consistently build reproducibly in the dozens of times I've build
> it...
> 
> 
>> shar actually makes the timestamps reproducible by always using the
>> mtime recorded in the archive instead of the time the files were unpacked.
>>
>> Something else is likely changing the mtime after the files are
>> unpacked. Some of these grids are used by the testsuite for example.
> 
> I will try to look into it further, but without really being familiar
> with the proj codebase (or even what proj itself does)... any additional
> very specific suggestions where to look next would definitely be
> appreciated!  :)

CMake's configure_file() is used to copy the .gsb & .gtx files from 
CMAKE_CURRENT_SOURCE_DIR to CMAKE_CURRENT_BINARY_DIR, that might be 
touching the mtimes too. See: data/CMakeLists.txt

Kind Regards,

Bas

-- 
  GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1