? - © Copyright 2012 - 2026, Jenkins Job Builder Maintainers. ? + © Copyright 2012 - 2025, Jenkins Job Builder Maintainers. ? Created using Sphinx 8 Patch attached that sources this value from SOURCE_DATE_EPOCH. [0] https://reproducible-builds.org/ Regards, -- ,''`. : :' : Chris Lamb `. `'` lamby at debian.org / chris-lamb.co.uk `- -------------- next part -------------- --- a/debian/patches/reproducible-build.patch 1970-01-01 01:00:00.000000000 +0100 --- b/debian/patches/reproducible-build.patch 2025-03-31 11:01:08.393654866 +0100 @@ -0,0 +1,28 @@ +Description: Make the build reproducible +Author: Chris Lamb +Last-Update: 2025-03-31 + +--- jenkins-job-builder-6.4.2.orig/doc/source/conf.py ++++ jenkins-job-builder-6.4.2/doc/source/conf.py +@@ -13,6 +13,7 @@ + + import os + import sys ++import time + import datetime + from jenkins_jobs.version import version_info as jenkins_jobs_version + +@@ -52,8 +53,12 @@ master_doc = "index" + + # General information about the project. + project = "Jenkins Job Builder" ++build_date = datetime.datetime.fromtimestamp( ++ int(os.environ.get('SOURCE_DATE_EPOCH', time.time())), ++ tz=datetime.timezone.utc, ++) + copyright = "2012 - {}, Jenkins Job Builder Maintainers".format( +- datetime.datetime.now().year ++ build_date.year + ) + + # The version info for the project you're documenting, acts as replacement for --- a/debian/patches/series 2025-03-31 10:47:39.413590873 +0100 --- b/debian/patches/series 2025-03-31 11:01:07.109648386 +0100 @@ -1 +1,2 @@ do-not-depends-on-pkg-resources.patch +reproducible-build.patch From lamby at debian.org Mon Mar 31 11:17:15 2025 From: lamby at debian.org (Chris Lamb) Date: Mon, 31 Mar 2025 11:17:15 +0100 Subject: Bug#1101741: isync: please make the build reproducible Message-ID: <174341615717.6720.16789259263403590382@b7e61f3d91b8> Source: isync Version: 1.5.1-1 Severity: wishlist Tags: patch User: reproducible-builds at lists.alioth.debian.org Usertags: timestamps X-Debbugs-Cc: reproducible-bugs at lists.alioth.debian.org Hi, Whilst working on the Reproducible Builds effort [0], we noticed that isync could not be built reproducibly. This is because the manual pages included the current date: ??? ./usr/share/man/man1/mbsync.1.gz ? ? @@ -1,14 +1,14 @@ ? ? .\" mbsync - mailbox synchronizer ? ? . ? ? -.TH mbsync 1 2025-03-13 "isync 1.5.1" "User Commands" ? ? +.TH mbsync 1 2026-04-15 "isync 1.5.1" "User Commands" ? ? . ? ? .SH NAME Patch attached that sources this value from SOURCE_DATE_EPOCH. [0] https://reproducible-builds.org/ Regards, -- ,''`. : :' : Chris Lamb `. `'` lamby at debian.org / chris-lamb.co.uk `- -------------- next part -------------- --- a/debian/patches/reproducible-build.patch 1970-01-01 01:00:00.000000000 +0100 --- b/debian/patches/reproducible-build.patch 2025-03-31 11:14:22.256947078 +0100 @@ -0,0 +1,15 @@ +Description: Make the build reproducible +Author: Chris Lamb +Last-Update: 2025-03-31 + +--- isync-1.5.1.orig/configure.ac ++++ isync-1.5.1/configure.ac +@@ -251,7 +251,7 @@ if test "x$have_macos_keychain" != xno; + AC_SUBST(KEYCHAIN_LIBS, ["-Wl,-framework,Security,-framework,CoreFoundation"]) + fi + +-RELEASE_DATE=`date -r $0 +%F` ++RELEASE_DATE=m4_esyscmd([date --utc --date="@${SOURCE_DATE_EPOCH:-$(date +%s)}" +%Y-%m-%d]) + AC_SUBST(RELEASE_DATE) + + AC_CONFIG_FILES([Makefile src/Makefile src/mbsync.1 src/mdconvert.1 isync.spec]) --- a/debian/patches/series 1970-01-01 01:00:00.000000000 +0100 --- b/debian/patches/series 2025-03-31 11:14:19.800927193 +0100 @@ -0,0 +1 @@ +reproducible-build.patch From lamby at debian.org Mon Mar 31 11:20:29 2025 From: lamby at debian.org (Chris Lamb) Date: Mon, 31 Mar 2025 11:20:29 +0100 Subject: Bug#1101742: python-pytest-shell-utilities: please make the build reproducible Message-ID: <174341635407.5791.5448883741403354146@8a42fd8642e8> Source: python-pytest-shell-utilities Version: 1.9.7-1 Severity: wishlist Tags: patch User: reproducible-builds at lists.alioth.debian.org Usertags: timestamps X-Debbugs-Cc: reproducible-bugs at lists.alioth.debian.org Hi, Whilst working on the Reproducible Builds effort [0], we noticed that python-pytest-shell-utilities could not be built reproducibly. This was because of two reasons. First, the documentation used the current build date instead of seeding it from SOURCE_DATE_EPOCH, and then the 'towncrier' Sphinx extension was adding its own date when adding stuff like "[UNRELEASED DRAFT]". Patch attached. [0] https://reproducible-builds.org/ Regards, -- ,''`. : :' : Chris Lamb `. `'` lamby at debian.org / chris-lamb.co.uk `- -------------- next part -------------- --- a/debian/patches/0002-Reproducible-build.patch 1970-01-01 01:00:00.000000000 +0100 --- b/debian/patches/0002-Reproducible-build.patch 2025-03-31 11:06:38.676581722 +0100 @@ -0,0 +1,34 @@ +Description: Make the build reproducible +Author: Chris Lamb +Last-Update: 2025-03-31 + +--- python-pytest-shell-utilities-1.9.7.orig/docs/conf.py ++++ python-pytest-shell-utilities-1.9.7/docs/conf.py +@@ -14,6 +14,7 @@ + import datetime + import os + import pathlib ++import time + import sys + + import pytest +@@ -45,7 +46,10 @@ dist = distribution("pytest-shell-utilit + + + # -- Project information ----------------------------------------------------- +-this_year = datetime.datetime.today().year ++this_year = datetime.datetime.fromtimestamp( ++ int(os.environ.get('SOURCE_DATE_EPOCH', time.time())), ++ tz=datetime.timezone.utc, ++).year + if this_year == 2021: + copyright_year = 2021 + else: +@@ -83,7 +87,6 @@ extensions = [ + "sphinx.ext.coverage", + "sphinx_copybutton", + "sphinxcontrib.spelling", +- "sphinxcontrib.towncrier", + ] + + # Add any paths that contain templates here, relative to this directory. --- a/debian/patches/series 2025-03-31 10:46:25.529222853 +0100 --- b/debian/patches/series 2025-03-31 10:57:49.640652686 +0100 @@ -1 +1,2 @@ 0001-Use-local-objects.inv-in-intersphinx-mapping.patch +0002-Reproducible-build.patch From lamby at debian.org Mon Mar 31 11:25:59 2025 From: lamby at debian.org (Chris Lamb) Date: Mon, 31 Mar 2025 11:25:59 +0100 Subject: Bug#1101743: oss4: please make the build reproducible Message-ID: <174341669613.22549.5281696713951690479@d475887df54f> Source: oss4 Version: 4.2-build2020-4 Severity: wishlist Tags: patch User: reproducible-builds at lists.alioth.debian.org Usertags: environment X-Debbugs-Cc: reproducible-bugs at lists.alioth.debian.org Hi, Whilst working on the Reproducible Builds effort [0], we noticed that oss4 could not be built reproducibly. This is because the embedded /usr/src/oss4.tar.bz2 tarball varied its contents based on the current user. Adding --owner=0, --group=0 and --numeric-owner to the tar(1) call fixes this. Patch attached. [0] https://reproducible-builds.org/ Regards, -- ,''`. : :' : Chris Lamb `. `'` lamby at debian.org / chris-lamb.co.uk `- -------------- next part -------------- --- a/debian/rules 2025-03-31 10:51:34.258765424 +0100 --- b/debian/rules 2025-03-31 11:22:56.328706071 +0100 @@ -56,7 +56,7 @@ cp debian/copyright build-tree/modules/oss4/debian/ cp debian/changelog build-tree/modules/oss4/debian/ chmod 755 build-tree/modules/oss4/debian/rules - cd build-tree/ && tar --mtime="@$(SOURCE_DATE_EPOCH)" --sort=name --mode=go=rX,u+rw,a-s -cjf oss4.tar.bz2 modules/ + cd build-tree/ && tar --mtime="@$(SOURCE_DATE_EPOCH)" --sort=name --mode=go=rX,u+rw,a-s --owner=0 --group=0 --numeric-owner -cjf oss4.tar.bz2 modules/ build/oss4-dkms:: stamp-source-oss4 sed -e 's#_VERSION_#$(UPSTREAM_VERSION)#' < debian/oss4-dkms.install.in > debian/oss4-dkms.install From ansgar at 43-1.org Wed Apr 2 20:41:39 2025 From: ansgar at 43-1.org (Ansgar =?UTF-8?Q?=F0=9F=99=80?=) Date: Wed, 02 Apr 2025 21:41:39 +0200 Subject: Bug#1072205: prevent re-using package versions for NMUs In-Reply-To: <174360039009.1060587.1045667824934462334@localhost> References: <171706060866.347506.10831597290300366561.reportbug@localhost> <174360039009.1060587.1045667824934462334@localhost> Message-ID: <4d52527c0b550f4a7ce716cfc6ded8da9482cde7.camel@43-1.org> Hi, regarding your analysis: I think you could just scan the .changes files as they list all *.deb files uploaded. Though very old changes only have MD5 hashes. They can be found in file://mirror.ftp-master.debian.org/srv/ftp-master.debian.org/queue/done Regarding your observation regarding bash not showing up in any Packages index: that can happen for (at least) two reasons. The snapshot service does not retrieve all Packages files. Or the package could have been superseded by a newer version before it was ever published in a dinstall run. Regards, Ansgar On Wed, 2025-04-02 at 15:26 +0200, Johannes Schauer Marin Rodrigues wrote: > Hi, > > On Thu, 30 May 2024 14:26:31 +0000 Holger Levsen > wrote: > > very "nice" find, josch! > > with the help of Holger and osuosl4 I have dug into this a bit more > and tried > to get some hard data about this problem. My idea was the following: > parse all > Packages files for all suites, all architectures and all components > for all > timestamps stored on snapshot.d.o and find packages with the same > name/arch/version tuple that have a different checksum. To this end, > I slightly > (less than 1000 lines of diff) patched the tooling at > https://salsa.debian.org/metasnap-team/metasnap.git?with the patch > that I > attached to this mail on top of > 1dadf2575160caf9467c4e21aa6c0a31ac10ffc2. > > After running that script for 3 months and downloading 189 GB of data > in 3.5 > Million requests (about 2 seconds for every request), we had a > database > (actually a git repository) of 48 GB that we can use to find > duplicates. It > took another 2 months to go through that data. I attached a graph > which shows > the number of duplicate name/arch/version triplets per timestamp. > Please note > the logarithmic y-axis. The total number of duplicates from 2005 > until 2024 is > 334335. > > Problem solved? Not so fast. Processing all Packages files will *not* > find the > original problem with bash. Why? Because according to the Packages > files from > snapshot.debian.org only one version of bash:arm64=5.2.15-2+b3 > exists, namely: > > MD5sum: 01ee4cfa3df78e7ff0dc156ff19e2c88 > SHA1:?? 1a0b12419b69a983bf22ac1d3d9f8bec725487b1 > SHA256: > 828ce0b4445921fff5b6394e74cce8296f3038d559845a3e82435b55ca6fcaa8 > > The other version never ended up in a Packages file even though it > was found in > the? /pool/main/b/bash directory in the snapshot of 2023-07-13 > 21:11:09 nearly > one year before the other version popped up. > > How can a package be in the pool directory but not in a Packages > file? No idea > but it shows that my method from above does not find a certain class > of > problems. We could find those by creating a fitting query against the > snapshot.d.o database. Apparently lw07 is DD accessible and has a > snapshot-guest service. So this is on my TODO list and Nicolas > Dandrimont > already offered to help with constructing an appropriate SQL query > during > MiniDebConf Hamburg this year. > > Lastly there is the problem of packages in incoming. Those packages > will be > used to build other packages that end up in the archive but they > might never > end up in the archive themselves. Thus, we might never know whether > one of > these packages violated the idea that the > packagename/architecture/version > triplet uniquely identifies a Debian binary package in the archive... > > Thanks! > > cheers, josch -------------- next part -------------- An HTML attachment was scrubbed... URL: From josch at debian.org Wed Apr 2 23:25:01 2025 From: josch at debian.org (Johannes Schauer Marin Rodrigues) Date: Thu, 03 Apr 2025 00:25:01 +0200 Subject: Bug#1072205: prevent re-using package versions for NMUs In-Reply-To: <4d52527c0b550f4a7ce716cfc6ded8da9482cde7.camel@43-1.org> References: <171706060866.347506.10831597290300366561.reportbug@localhost> <174360039009.1060587.1045667824934462334@localhost> <4d52527c0b550f4a7ce716cfc6ded8da9482cde7.camel@43-1.org> Message-ID: <174363270188.5564.8524691288965726855@localhost> Hi Ansgar, thank you for chiming in! Quoting Ansgar ? (2025-04-02 21:41:39) > regarding your analysis: I think you could just scan the .changes files as > they list all *.deb files uploaded.> Though very old changes only have MD5 > hashes. > > They can be found in > file://mirror.ftp-master.debian.org/srv/ftp-master.debian.org/queue/done are those the changes files uploading *.debs produced by the buildds? Does that include *.changes files from package builds which only ever affected incoming because there was a new build of the same package before the next dinstall? Looking at those .changes files, it indeed seems that the binNMUs were made by the buildds, so those are not maintainer changes files. > Regarding your observation regarding bash not showing up in any Packages > index: that can happen for (at least) two reasons. The snapshot service does > not retrieve all Packages files. You mean the snapshot service was buggy and forgot to retrieve an arm64 Packages file? Is that likely? Lets have a look: http://snapshot.debian.org/package/bash/5.2.15-2/#bash_5.2.15-2:2b:b3 We have two different copies of bash:arm64=5.2.15-2+b3: 20230713T211109Z until 20230813T092534Z 01ee4cfa3df78e7ff0dc156ff19e2c88 597e85b2fdc7fe9da3f3215030e7c3dead3f464c 828ce0b4445921fff5b6394e74cce8296f3038d559845a3e82435b55ca6fcaa8 http://snapshot.debian.org/archive/debian/20230713T211109Z/pool/main/b/bash/bash/bash_5.2.15-2%2Bb3_arm64.deb 20240331T210805Z until 20240402T025937Z 2f8db1d53e5b2e6292b3c408d55ae5de 1a0b12419b69a983bf22ac1d3d9f8bec725487b1 58000b621cceafd717ed0bf4cb428ba1220c0e1993620bde8ad8a3ccc765780b http://snapshot.debian.org/archive/debian/20240331T210805Z/pool/main/b/bash/bash_5.2.15-2%2Bb3_arm64.deb I had a look at the changes files stored in /srv/ftp-master.debian.org/queue/done/2024/03.tar.xz and indeed there I found 03/31/bash_5.2.15-2+b3_arm64-buildd.changes which is the binNMU to Bookworm. The same tarball contains another changes file for bash from eight hours later called 03/31/bash_5.2.15-2+b7_arm64-buildd.changes. Maybe it is indeed possible that snapshot.d.o skipped the one Packages for Bookworm file that should've contained the former version? > Or the package could have been superseded by a newer version before it was > ever published in a dinstall run. But hashes of packages that were never published by a dinstall should be part of the changes file on coccia, right? That would be really nice! A year of changes files seems to be just around 2.6 GB which is really tiny compared to the data I was processing before. Is this really the silver bullet that solves all my problems with finding these hash collisions? Can I just scp those tar.xz files from coccia to my local box and process them to my heart's content? Is there a catch? Thanks! cheers, josch -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: signature URL: From ansgar at 43-1.org Thu Apr 3 09:14:28 2025 From: ansgar at 43-1.org (Ansgar =?UTF-8?Q?=F0=9F=99=80?=) Date: Thu, 03 Apr 2025 10:14:28 +0200 Subject: Bug#1072205: prevent re-using package versions for NMUs In-Reply-To: <174363270188.5564.8524691288965726855@localhost> References: <171706060866.347506.10831597290300366561.reportbug@localhost> <174360039009.1060587.1045667824934462334@localhost> <4d52527c0b550f4a7ce716cfc6ded8da9482cde7.camel@43-1.org> <174363270188.5564.8524691288965726855@localhost> Message-ID: Hi, On Thu, 2025-04-03 at 00:25 +0200, Johannes Schauer Marin Rodrigues wrote: > Quoting Ansgar ? (2025-04-02 21:41:39) > > regarding your analysis: I think you could just scan the .changes files as > > they list all *.deb files uploaded.> Though very old changes only have MD5 > > hashes. > > > > They can be found in > > file://mirror.ftp-master.debian.org/srv/ftp-master.debian.org/queue/done > > are those the changes files uploading *.debs produced by the buildds? > > Does that include *.changes files from package builds which only ever affected > incoming because there was a new build of the same package before the next > dinstall? > > Looking at those .changes files, it indeed seems that the binNMUs were made by > the buildds, so those are not maintainer changes files. All regular uploads to the archive should come with a .changes files and these should be all, that is, for both maintainer uploads and buildd uploads. (Note that historic buildd uploads were also signed by a DD key.) It does not cover uploads to the security archive, but packages synced from the security archive to the main archive should be recorded there. It also does not include rejected packages (e.g., for policy queues before *-proposed-updates, new or even before). But for your use case these are probably not relevant. There is a mechanism in DAK to import packages without .changes files, but that shouldn't be used much. The only regular use case is adding source packages for Built-Using to the security archive. > > Regarding your observation regarding bash not showing up in any Packages > > index: that can happen for (at least) two reasons. The snapshot service does > > not retrieve all Packages files. > > You mean the snapshot service was buggy and forgot to retrieve an arm64 > Packages file? Is that likely? Lets have a look: There is no technical mechanism to ensure files end up in snapshot.d.o. If the service does not import them on time, they will not end up in snapshot.d.o; the archive does currently not retain files until the snapshot service picked them up. There have been times in the past where snapshot.d.o was not always reliably importing files, see for example [1]. [1]: https://lists.debian.org/msgid-search/169097599149.3513878.11315997455981479167 at localhost I've not checked if this was the case for the particular instance you mentioned. > > Or the package could have been superseded by a newer version before it was > > ever published in a dinstall run. > > But hashes of packages that were never published by a dinstall should be part > of the changes file on coccia, right? That would be really nice! A year of > changes files seems to be just around 2.6 GB which is really tiny compared to > the data I was processing before. Yes, they should be. Only the .changes files might be even less data: the last years also include .buildinfo files. > Can I just scp those tar.xz files from coccia to my local box and process them > to my heart's content? Is there a catch? I think that is fine. The equivalent data from the security archive is not on a DD-accessible mirror, but if you find it useful, we could probably make them available to you (at least provide a one-time dump). As this is only about published packages, there is no need for confidentiality. Ansgar