Bug#952762: openstack-pkg-tools: please make the build reproducible

[Chris Lamb]

Source: openstack-pkg-tools
Version: 108
Severity: wishlist
Tags: patch
User: reproducible-builds at lists.alioth.debian.org
Usertags: toolchain
X-Debbugs-Cc: reproducible-bugs at lists.alioth.debian.org

Hi,

Whilst working on the Reproducible Builds effort [0] we noticed that
openstack-pkg-tools is causing other packages to be built in an
unreproducible manner.

In particular, the "/usr/bin/pkgos-dh_auto_install" script may nondeterministically create packages with differing shebangs and binary dependencies. For example, this is from src:redfishtool:

│ -#!/usr/bin/python3.7
│ +#!/usr/bin/python3.8

[…]

│ │ │ │ -Depends: python3-requests, python3.8:any, python3:any
│ │ │ │ +Depends: python3-requests, python3.7:any, python3:any

§

This is caused by a number of layered reasons. First, we are building
all supported Python versions (eg. Python 3.7 and Python 3.8) in
separate directories but then seqeuentially installing them to the
same destination, debian/${TARGET_DIR}.

However, this causes problems because if latter installations complete
in less than one second, distutils may decide to skip copying files in
the shared destination as it incorrectly believes them to be up-to-
date. This will result in a package arbitrarily containing scripts
with different version shebangs depending on the approximate total
execution speed of installation. This is, needless to say,
nondeterminstic.

For example, if we build for both Python 3.7 and Python 3.8 but the
installation of the latter occurs within the same wall clock second of
the former, the Python 3.8 version will not overwrite the Python 3.7
verison and lead to a shebang of #!/usr/bin/python3.7 … whilst if it
does not occur within the same second, the shebang will be overwritten
to #!/usr/bin/python3.8.

A patch is attached that passes --force to `setup.py install [..]`
which will avoid the underlying calls to distutils's `dep_util.newer`
and thus will always update.

  [0] https://reproducible-builds.org/


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby at debian.org / chris-lamb.co.uk
       `-
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: openstack-pkg-tools.diff.txt
URL: <http://alioth-lists.debian.net/pipermail/reproducible-bugs/attachments/20200228/c9f0e9bd/attachment.txt>