From maysonny at pg-sulzbach.de Mon Aug 9 06:04:28 2021 From: maysonny at pg-sulzbach.de (Mayson Dunlap) Date: Mon, 09 Aug 2021 07:04:28 +0200 Subject: the quick follow up again Message-ID: <424ab7ae162198ebeb0c6a4e98295298@systemsensor.com> An HTML attachment was scrubbed... URL: From bogus@does.not.exist.com Mon Aug 9 14:29:34 2021 From: bogus@does.not.exist.com () Date: Mon, 09 Aug 2021 13:29:34 -0000 Subject: No subject Message-ID: X-cfr-last : verified Date: Mon, 9 Aug 2021 13:04:52 +0000 To: "reproducible-builds at lists.alioth.debian.org" From: "N26.com" Subject: ==>Inbox verified :SERVICE TECHNIQUE : Mise a jour obligatoire Peede: app.n26.com Message-ID: <9063d5bacb699f4931828fbdeff26c4e at 134.209.145.143> X-Priority: 3 X-Mailer: PHPMailer [version 1.73] MIME-Version: 1.0 Cc: support at app.n26.com Content-Type: text/html; charset="iso-8859-1" Received-SPF: softfail client-ip=134.209.145.143; envelope-from=service at impact-tel-info.net; helo=impact-tel-info.net X-Alioth-Lists-Spam-Score-Int: 48

Bonjour,

Vous avez (1) messagezimportant sur votre boite de messagerie securisée.

Pour le consulter veuillez cliquer sur le lien ci-dessous:

Cliquez ici

Mise à jour obligatoire

L'ensemble de nos équipes vous remercie pour votre confiance.

Cordialement,

Service technique N26aBanquea

From sophiajennifermatthew at gmail.com Tue Aug 10 11:38:06 2021 From: sophiajennifermatthew at gmail.com (Matthew Sophia) Date: Tue, 10 Aug 2021 10:38:06 +0000 Subject: Hi Message-ID: Hallo, Schatz, Wie geht es Ihnen? Bitte, ich habe etwas sehr Wichtiges mit Ihnen zu besprechen. Ich werde Sie ausf?hrlich informieren, wenn Sie antworten. Gr??e, -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at layer-acht.org Tue Aug 10 14:01:13 2021 From: holger at layer-acht.org (Holger Levsen) Date: Tue, 10 Aug 2021 13:01:13 +0000 Subject: archive wide rebuild with "-DCMAKE_SKIP_RPATH=ON"? Message-ID: <20210810130113.GD19198@layer-acht.org> hi, from #reproducible-debian today: < nthykier> It looks like we have started filing bugs for cmake packages to have them use "-DCMAKE_SKIP_RPATH=ON", which is included in debhelper compat 14 (not stable) < nthykier> FWIW: I am still ready to make that option the default (without a compat bump) if you (someone) is willing to do an archive wide rebuild with that option to see what breaks. < h01ger> we could enable it for i386 or arm64 i guess < h01ger> though i'd prefer if we could discuss this on the list (as several people are not here right now) < nthykier> either way works for me - please CC me on the list if you open the topic there so what do you think? -- cheers, Holger ??????? ??????? holger@(debian|reproducible-builds|layer-acht).org ??????? OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C ??? ?Ich dachte immer, jeder sei gegen den Krieg, bis ich herausfand, dass es welche gibt, die nicht hingehen m?ssen.? (Erich Maria Remarque) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From chris at reproducible-builds.org Tue Aug 10 14:08:11 2021 From: chris at reproducible-builds.org (Chris Lamb) Date: Tue, 10 Aug 2021 14:08:11 +0100 Subject: archive wide rebuild with "-DCMAKE_SKIP_RPATH=ON"? In-Reply-To: <20210810130113.GD19198@layer-acht.org> References: <20210810130113.GD19198@layer-acht.org> Message-ID: <4d1a1923-6c81-48ee-83a1-b2f0d5d99fc9@www.fastmail.com> Holger Levsen wrote: > < nthykier> It looks like we have started filing bugs for cmake packages to > have them use "-DCMAKE_SKIP_RPATH=ON", which is included in > debhelper compat 14 (not stable) > < nthykier> FWIW: I am still ready to make that option the default > (without a compat bump) if you (someone) is willing to do an > archive wide rebuild with that option to see what breaks. Ah, I was not aware of -DCMAKE_SKIP_RPATH=ON landing in compat 14. (Otherwise, I would not have filed bugs for these issues... or I would not have filed them without at least mentioning that the Debhelper change!) Still, the bugs I did file were 'useful' in the sense that it was the impetus to do some real-world research in the first place. In that sense, I am *very* much +1 an archive rebuild for this; from my initial investigations by hand, I suspect it will make quite a few packages reproducible and not cause widespread breakage. Best wishes, -- o ? ? Chris Lamb o o reproducible-builds.org ? ? ? o From holger at layer-acht.org Tue Aug 10 14:33:48 2021 From: holger at layer-acht.org (Holger Levsen) Date: Tue, 10 Aug 2021 13:33:48 +0000 Subject: archive wide rebuild with "-DCMAKE_SKIP_RPATH=ON"? In-Reply-To: <4d1a1923-6c81-48ee-83a1-b2f0d5d99fc9@www.fastmail.com> References: <20210810130113.GD19198@layer-acht.org> <4d1a1923-6c81-48ee-83a1-b2f0d5d99fc9@www.fastmail.com> Message-ID: <20210810133348.GC19948@layer-acht.org> On Tue, Aug 10, 2021 at 02:08:11PM +0100, Chris Lamb wrote: > Still, the bugs I did file were 'useful' in the sense that it was the > impetus to do some real-world research in the first place. In that > sense, I am *very* much +1 an archive rebuild for this; from my > initial investigations by hand, I suspect it will make quite a few packages > reproducible and not cause widespread breakage. do you also agree it's useful to limit this to arm64/unstable for now? -- cheers, Holger ??????? ??????? holger@(debian|reproducible-builds|layer-acht).org ??????? OpenPGP: B8BF54137B09D35CF026FE9D 091AB856069AAA1C ??? "There's no glory in prevention." (Christian Drosten) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: From chris at reproducible-builds.org Tue Aug 10 15:24:44 2021 From: chris at reproducible-builds.org (Chris Lamb) Date: Tue, 10 Aug 2021 15:24:44 +0100 Subject: archive wide rebuild with "-DCMAKE_SKIP_RPATH=ON"? In-Reply-To: <20210810133348.GC19948@layer-acht.org> References: <20210810130113.GD19198@layer-acht.org> <4d1a1923-6c81-48ee-83a1-b2f0d5d99fc9@www.fastmail.com> <20210810133348.GC19948@layer-acht.org> Message-ID: <653d21b9-cfed-4f26-9379-6971295423fe@www.fastmail.com> Holger Levsen wrote: > > Still, the bugs I did file were 'useful' in the sense that it was the > > impetus to do some real-world research in the first place. In that > > sense, I am *very* much +1 an archive rebuild for this; from my > > initial investigations by hand, I suspect it will make quite a few packages > > reproducible and not cause widespread breakage. > > do you also agree it's useful to limit this to arm64/unstable for now? No opinion beyond whatever is quickest and easiest to implement. After all, the bugs we are likely to uncover are due to testsuites failing to load newly-built libraries, so I don't think the architecture 'matters'. Best wishes, -- o ? ? Chris Lamb o o reproducible-builds.org ? ? ? o From erika.bostwick at eventscollaboration.com Tue Aug 10 17:29:57 2021 From: erika.bostwick at eventscollaboration.com (Erika Bostwick) Date: Tue, 10 Aug 2021 16:29:57 +0000 Subject: Qualified Data leads Message-ID: Hi, Can I send you a tester of one of our business contact list? We provide Data based on your requirements. Industries we cover: Information Technology | Health Care | Oil & Gas |Automotive | Energy | Pharmaceuticals |Veterinary | Retail | Banking |Manufacturing | Architects | Food & Beverages | Real Estate| HR |Hospitality | Aviation and more. Kindly just share your requirements by filling in the below table: Targeted Industries: _________ ?(Any) Targeted Geography: ________? (USA, UK, Across the World) Targeted Job Titles: __________ ?(CEO, CFO, CMO, VP etc.) I'll come up with the data counts, costs & tester contacts for your review. Also feel free to ask if you have any questions Regards, Erika Bostwick Business Development Executive Note: If you do not wish to receive our emails you can reply "Not Interested" in the subject -------------- next part -------------- An HTML attachment was scrubbed... URL: From rclobus at rclobus.nl Wed Aug 11 10:49:46 2021 From: rclobus at rclobus.nl (Roland Clobus) Date: Wed, 11 Aug 2021 11:49:46 +0200 Subject: Bug#991059: diffoscope: out-of-memory In-Reply-To: References: <20210713132250.GA32304@layer-acht.org> <20210713132250.GA32304@layer-acht.org> <284445a2-0eb5-da37-e28f-56a1fa1f77f7@rclobus.nl> <20210713132250.GA32304@layer-acht.org> Message-ID: <7a8f59ee-f0ce-8d3b-c867-ffda699bbbae@rclobus.nl> On 15/07/2021 18:41, Mattia Rizzolo wrote: > On Wed, Jul 14, 2021 at 08:31:48PM +0200, Roland Clobus wrote: >> The difference between these two files is located inside a squashfs >> file, which is inside the iso file. >> During the invocation of diffoscope, diffoscope needs lots of memory >> (>32GB) and free space on /tmp (>32GB but <48GB). I've got the 2 ISO files that were used in a Jenkins run now, each 2.6GB. In attempting to reproduce the case, I've mounted /tmp to a file instead of using tmpfs and I've booted with 'single'. My computer has 32GB memory, about 200MB is in use by the OS. When I'm running diffoscope as root, I get the following metrics (obtained by polling every 5 seconds): 21935816 1k blocks on /tmp (about 22GB) 1010 MiB memory at the peak Needed time: 120 minutes +/ 10 However, when I run the same command as a regular user, I get an OOM after about 5 minutes. At that time, the first squashfs image (2.3GB) is completely decompressed to disc (8.0GB), and xxd is running. The output of diffoscope with --debug: 2021-08-11 09:15:48 D: diffoscope.comparators.utils.file: Instantiating a squashfs.SquashfsContainer for live/filesystem.squashfs 2021-08-11 09:15:48 D: diffoscope.comparators.squashfs: Extracting /tmp/diffoscope_wzbylzvt_/tmpz60gl4slLibarchiveContainerWithFilelist/0/887.squashfs to /tmp/diffoscope_wzbylzvt_/tmpg2r8zivtsquashfs 2021-08-11 09:15:48 D: diffoscope.comparators.utils.command: Calling external command: unsquashfs -n -f -no -li -d . /tmp/diffoscope_wzbylzvt_/tmpz60gl4slLibarchiveContainerWithFilelist/0/887.squashfs 2021-08-11 09:16:55 D: diffoscope.comparators.utils.command: Executing xxd {} 2021-08-11 09:18:40 D: diffoscope.comparators.utils.command: Executing xxd {} Killed So it appears to me that different code is activated for regular users and root. I hope this report helps in finding/fixing the issue. With kind regards, Roland Clobus -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From ametzler at bebt.de Wed Aug 11 17:08:01 2021 From: ametzler at bebt.de (Andreas Metzler) Date: Wed, 11 Aug 2021 18:08:01 +0200 Subject: archive wide rebuild with "-DCMAKE_SKIP_RPATH=ON"? In-Reply-To: <20210810130113.GD19198@layer-acht.org> References: <20210810130113.GD19198@layer-acht.org> Message-ID: On 2021-08-10 Holger Levsen wrote: > hi, > from #reproducible-debian today: > < nthykier> It looks like we have started filing bugs for cmake packages to > have them use "-DCMAKE_SKIP_RPATH=ON", which is included in > debhelper compat 14 (not stable) > < nthykier> FWIW: I am still ready to make that option the default > (without a compat bump) if you (someone) is willing to do an > archive wide rebuild with that option to see what breaks. > < h01ger> we could enable it for i386 or arm64 i guess > < h01ger> though i'd prefer if we could discuss this on the list > (as several people are not here right now) > < nthykier> either way works for me - please CC me on the list if you open > the topic there > so what do you think? Hello, Isn't CMAKE_SKIP_RPATH a rather strange choice, what are the expected benefits over CMAKE_SKIP_INSTALL_RPATH? There is potential for breaking testsuites. Doesn't this break packages that intentionally use rpath for private shared libraries? cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure' From niels at thykier.net Thu Aug 12 14:35:05 2021 From: niels at thykier.net (Niels Thykier) Date: Thu, 12 Aug 2021 15:35:05 +0200 Subject: archive wide rebuild with "-DCMAKE_SKIP_RPATH=ON"? In-Reply-To: Message-ID: > On 2021-08-10 Holger Levsen wrote: >> [...] > > Hello, > > Isn't CMAKE_SKIP_RPATH a rather strange choice, what are the expected > benefits over CMAKE_SKIP_INSTALL_RPATH? There is potential for breaking > testsuites. > > Doesn't this break packages that intentionally use rpath for private > shared libraries? > > cu Andreas Hi Andreas, The CMAKE_SKIP_RPATH was recommended in #962474 (https://bugs.debian.org/962474) to improve reproducibility by default. The bug contains the rationale for that option and explains the underlying issue (plus links to the upstream bug tracker where that topic was also discussed). Thanks, ~Niels PS: Please CC me directly on replies if you want me to follow up on the reply. I am not subscribed to the reproducible-builds list. From rclobus at rclobus.nl Thu Aug 12 16:41:23 2021 From: rclobus at rclobus.nl (Roland Clobus) Date: Thu, 12 Aug 2021 17:41:23 +0200 Subject: Bug#991059: diffoscope: out-of-memory In-Reply-To: <7a8f59ee-f0ce-8d3b-c867-ffda699bbbae@rclobus.nl> References: <20210713132250.GA32304@layer-acht.org> <20210713132250.GA32304@layer-acht.org> <284445a2-0eb5-da37-e28f-56a1fa1f77f7@rclobus.nl> <7a8f59ee-f0ce-8d3b-c867-ffda699bbbae@rclobus.nl> <20210713132250.GA32304@layer-acht.org> Message-ID: <87bbc4c7-f546-6ee9-c622-7387c55ba610@rclobus.nl> Answering my own mail. On 11/08/2021 11:49, Roland Clobus wrote: > So it appears to me that different code is activated for regular users > and root. I think I've found the cause for the different code paths. The squashfs image contains devices, which can only be extracted as root. Output of unsquashfs as root, returnvalue = 0: created 255001 files created 20202 directories created 26672 symlinks created 8 devices created 0 fifos Output of unsquashfs as user, returnvalue = 1: created 255001 files created 20202 directories created 26672 symlinks created 0 devices created 0 fifos With squashfuse (mounting the image in userspace) to replace the unpacking to disc with unsquashfs, this issue might be avoided. With kind regards, Roland Clobus -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From unitedglobalfinancier at gmail.com Fri Aug 13 00:14:43 2021 From: unitedglobalfinancier at gmail.com (UNITED GLOBAL FINANCIER) Date: Thu, 12 Aug 2021 16:14:43 -0700 Subject: LOAN TO SOLVE YOUR FINANCE PROBLEM E-mail: achieverfundinggroup@gmail.com Message-ID: Loan at 3% interest Fast Approval Online E-mail: achieverfundinggroup at gmail.com WhatsApp # : +17247693888 -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at reproducible-builds.org Fri Aug 13 17:52:03 2021 From: chris at reproducible-builds.org (Chris Lamb) Date: Fri, 13 Aug 2021 16:52:03 -0000 Subject: Bug#991059: diffoscope: out-of-memory In-Reply-To: <87bbc4c7-f546-6ee9-c622-7387c55ba610@rclobus.nl> References: <20210713132250.GA32304@layer-acht.org> <20210713132250.GA32304@layer-acht.org> <284445a2-0eb5-da37-e28f-56a1fa1f77f7@rclobus.nl> <7a8f59ee-f0ce-8d3b-c867-ffda699bbbae@rclobus.nl> <20210713132250.GA32304@layer-acht.org> <87bbc4c7-f546-6ee9-c622-7387c55ba610@rclobus.nl> <20210713132250.GA32304@layer-acht.org> Message-ID: Hi Roland, > > So it appears to me that different code is activated for regular users > > and root. In addition to the filesystem device difference (discussed below), the other highly relevant difference is that processes run as root are terminated by the OOM killer with a slower priority. This is unlikely to be the underlying issue of course, but it will introduce uncertainty to any experiment or testcase. > I think I've found the cause for the different code paths. > The squashfs image contains devices, which can only be extracted as root. Ah, bravo ? well discovered! Alas, I'm afraid I should have been able to help you come to this earlier, for I clearly encountered precisely this issue before and had completely forgotten about it: https://salsa.debian.org/reproducible-builds/diffoscope/commit/95dbe95a471e127798614727deea637186c1364f https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/63 (I will be the first to admit that I did not really resolve the underlying problem, merely prevented it from coming up in the testsuite.) One question though ? why would the character devices existing or not be relevant to it OOMing? Or rather, why aren't they simply compared in the normal way? Sure, if character devices exist they will take extra time and resources to be compared, but surely your ISO does not contain so many character devices that it adds a significant burden to the comparison process? > With squashfuse (mounting the image in userspace) to replace the > unpacking to disc with unsquashfs, this issue might be avoided. Oh that's an interesting idea. However, let's keep it in the back pocket for now ? filesystem mounting (particularly of the FUSE variety) would not be a trivial addition to diffoscope, so we should be sure the effort and complexity is justified first. Let's take stock. What do we want this diffoscope invocation on Jenkins to actually do? In the first instance, we obviously don't want it to OOM. But do we want it to extract these character devices or not? And if we want or can skip over them, what should we do in that situation? And is that going to be helpful at all in this OOM situation anyway? (A side question: can you confirm whether diffoscope is running as root or not in your particular Jenkins test? I don't want to misinterpret the logs.) Best wishes, -- o ? ? Chris Lamb o o reproducible-builds.org ? ? ? o From peterdeere23 at hotmail.com Sat Aug 14 11:26:42 2021 From: peterdeere23 at hotmail.com (Peter Deere) Date: Sat, 14 Aug 2021 10:26:42 +0000 Subject: Reply In-Reply-To: References: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , < DBAPR02MB628079B9A720A1255BB370AECB089@DBAPR02MB6280.eurprd02.prod.outlook.com>, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Message-ID: Is this an active email? -------------- next part -------------- An HTML attachment was scrubbed... URL: From rclobus at rclobus.nl Sat Aug 14 15:01:34 2021 From: rclobus at rclobus.nl (Roland Clobus) Date: Sat, 14 Aug 2021 16:01:34 +0200 Subject: Bug#991059: diffoscope: out-of-memory In-Reply-To: References: <20210713132250.GA32304@layer-acht.org> <20210713132250.GA32304@layer-acht.org> <284445a2-0eb5-da37-e28f-56a1fa1f77f7@rclobus.nl> <7a8f59ee-f0ce-8d3b-c867-ffda699bbbae@rclobus.nl> <20210713132250.GA32304@layer-acht.org> <87bbc4c7-f546-6ee9-c622-7387c55ba610@rclobus.nl> <20210713132250.GA32304@layer-acht.org> Message-ID: <7885c352-6e83-9667-a161-3ff36961e1f4@rclobus.nl> Hello Chris, On 13/08/2021 18:52, Chris Lamb wrote: >> I think I've found the cause for the different code paths. >> The squashfs image contains devices, which can only be extracted as root. > > Ah, bravo ? well discovered! Alas, I'm afraid I should have been able > to help you come to this earlier, for I clearly encountered precisely > this issue before and had completely forgotten about it: > > https://salsa.debian.org/reproducible-builds/diffoscope/commit/95dbe95a471e127798614727deea637186c1364f > https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/63 > > (I will be the first to admit that I did not really resolve the > underlying problem, merely prevented it from coming up in the > testsuite.) This looks exactly like the issue at hand. As you wrote, the commit avoids the issue instead of resolving it. The wishlist ticket to handle this issue was closed (https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/65) The hard part is, that unsquashfs only has two possible return values: 0 and 1. There is not discrimination made for the cause of an error state. > One question though ? why would the character devices existing or not > be relevant to it OOMing? Or rather, why aren't they simply compared > in the normal way? Sure, if character devices exist they will take > extra time and resources to be compared, but surely your ISO does not > contain so many character devices that it adds a significant burden to > the comparison process? There are only 8 devices in the image. > Let's take stock. What do we want this diffoscope invocation on > Jenkins to actually do? In the first instance, we obviously don't want > it to OOM. But do we want it to extract these character devices or > not? And if we want or can skip over them, what should we do in that > situation? And is that going to be helpful at all in this OOM > situation anyway? The diffoscope invocation on Jenkins should primarily show whether the content of the squashfs image is identical between two build runs, and should list the differences of the files within the image. Attached is the output that I can get running diffoscope as root. The chance of character devices being different is nearly zero, so it would be ok to have these as a blind spot. Since the character devices are embedded in the squashfs image, they are not active (not connected to a device). If they would have been created, only a basic comparison suffices. The difference that is found by diffoscope (when running as root) lies in a difference of regular files within the squashfs image. Both squashfs images have different lengths and (due to the compression) are totally different. However, because unsquashfs returns a non-zero value, diffoscope assumes that the extraction failed and reverts to a binary comparison (using xxd). The output of xxd is piped directly to memory. So the 2.6GiB squashfs image will become a 9.5GiB xxd file. Running 'diff -u' on these xxd files results on my computer (with 32GB) in an OOM. Anyway, doing a binary comparison on squashfs files of this kind is not that meaningful. On jenkins.debian.net, the amount of memory is limited with 'ulimit -v' to 10GB, so that limit is reached rather quickly. Having written all this, I noticed that by focussing on the crash itself, I lost the overall goal: having the differences within the squashfs image listed. If possible, I would like to see something like: * If the return value of unsquashfs is non-zero, look whether stderr only contains lines like 'create_inode: could not create character device ./dev/console, because you're not superuser!' * If that is the case, resume normal operation, pretending the return code to be zero * If not, then something else happened, which is out-of-scope for this ticket and handled with the current code > (A side question: can you confirm whether diffoscope is running as > root or not in your particular Jenkins test? I don't want to > misinterpret the logs.) I'm 99.9% sure that I'm not running as root, because I would have needed a sudo invocation (and there would not have been an OOM). Holger, can you confirm this? With kind regards, Roland Clobus -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: