Bug#1035704: proj: reproducible-builds: timezone-dependent timestamps in .gsb/.gtx files

[Sebastiaan Couwenberg]

On 5/28/23 23:38, Vagrant Cascadian wrote:
> On 2023-05-08, Vagrant Cascadian wrote:
>> On 2023-05-09, Sebastiaan Couwenberg wrote:
>>> On 5/9/23 05:47, Sebastiaan Couwenberg wrote:
>>>> On 5/8/23 22:43, Vagrant Cascadian wrote:
>>>>> On 2023-05-08, Sebastiaan Couwenberg wrote:
>>>>>> On 5/8/23 02:07, Vagrant Cascadian wrote:
>>>>>>> The attached patch fixes this by not touching these files during the
>>>>>>> build process.
>>>>>>
>>>>>>    From shar(1):
>>>>>>
>>>>>> "
>>>>>>      -m, --no-timestamp
>>>>>>             do not restore modification times.
> ...
>>>>>> That should be used when generating the archives instead of your patch
>>>>>> to not have the mtimes touched when unpacking the archives.
> ...
>>>>>> But the diffoscope-results only show a few of the files from the shar
>>>>>> archives with different mtimes, and all of them get touched when
>>>>>> unpacking the archive just before the configure target is executed.
>>>>>
>>>>> Well, I too was perplexed why other files were not affected, but it is
>>>>> consistently those .gsb and .gtx files, and the submitted patch allows
>>>>> to consistently build reproducibly in the dozens of times I've build
>>>>> it...
>>>>>
>>>>>
>>>>>> shar actually makes the timestamps reproducible by always using the
>>>>>> mtime recorded in the archive instead of the time the files were
>>>>>> unpacked.
>>>>>>
>>>>>> Something else is likely changing the mtime after the files are
>>>>>> unpacked. Some of these grids are used by the testsuite for example.
>>>>>
>>>>> I will try to look into it further, but without really being familiar
>>>>> with the proj codebase (or even what proj itself does)... any additional
>>>>> very specific suggestions where to look next would definitely be
>>>>> appreciated!  :)
>>>>
>>>> CMake's configure_file() is used to copy the .gsb & .gtx files from
>>>> CMAKE_CURRENT_SOURCE_DIR to CMAKE_CURRENT_BINARY_DIR, that might be
>>>> touching the mtimes too. See: data/CMakeLists.txt
>>
>> Thanks, that is definitely worth taking a look at...
> ...
>> Will try to poke at it more tomorrow...
> 
> I had no luck with poking at that approach... though did not have great
> ideas what to even try there.
> 
> That said, I think it really is the touch commands in debian/datumgrids*
> as touch's timestamp modification is timezone dependent in many cases...
> 
> The attached patch fixes this by setting the TZ=UTC as an environment
> variable in the debian/datumgrids*.shar files.
> 
> I also had success with a patch where the touch calls are done with
> --date=${SOURCE_DATE_EPOCH} which also worked for me (as touch assumes
> to be TZ=UTC in this case)... if that would be preferable, I can also
> provide a patch for that.

Patching the shar files is not ideal, when their content is modified 
these changes will be lost.

shar/unshar should be more likely be patched.

Does TZ=UTC also work when set in the environment? If so, that could be 
passed to the unshar commands in d/rules.

Kind Regards,

Bas

-- 
  GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146  50D1 6750 F10A E88D 4AF1