Installed-Build-Depends lack architecture qualification
Helmut Grohne
helmut at subdivi.de
Sat Sep 24 22:58:28 BST 2022
Control: tags -1 + patch
Hi Guillem, cross folks and reproducible folks,
On Thu, Aug 24, 2017 at 09:45:39PM +0200, Helmut Grohne wrote:
> while looking into a .buildinfo file, I noticed that
> Installed-Build-Depends are useless beyond #871494: They lack
> architecture qualification. Thus there is no way to figure out for which
> architecture to install which package. The installation set cannot be
> reproduced.
>
> I believe that a good solution here would be to simply arch qualify
> every package and then drop all :$DEB_BUILD_ARCH qualifications. For
> native builds there won't be a difference. For cross builds, the field
> becomes useful. I am proposing DEB_BUILD_ARCH rather than DEB_HOST_ARCH
> here, because essential will always be installed for the build
> architecture and thus poses significant chunk of packages.
I've implemented the proposed solution and seek review from both cross
and reproducible folks. You'll find a patch attached.
In order to get some confidence in this patch, I've performed some tests
and compared .buildinfo files using dpkg with/without this patch. I
chose hostname as a very simple example and libxml2 as an example that
involves dependencies on virtual packages, dependencies on Multi-Arch:
foreign packages, :any-annotated dependencies on Multi-Arch: allowed
packages as well as normal dependencies on Multi-Arch: same packages.
For a native build of hostname and libxml2, only the dpkg versions
differ.
For a cross build of hostname, additionally a number of dependencies
become duplicated with a :$DEB_HOST_ARCH qualifier.
For a cross build of libxml2, additionally the libicu-dev and
liblzma-dev dependencies become arch qualified and their unqualified
versions go away.
These results all look correct to me. Given that the native builds
didn't differ (beyond dpkg versions), the risk of regressions seems
fairly low to me.
What still is not included is the cross toolchain. The long-term
solution to this problem is including build-essential and
build-essential:native once build-essential has become Multi-Arch: same.
So let's not fix that here.
Helmut
-------------- next part --------------
diff --minimal -Nru dpkg-1.21.9/debian/changelog dpkg-1.21.9+nmu1/debian/changelog
--- dpkg-1.21.9/debian/changelog 2022-07-01 11:25:58.000000000 +0200
+++ dpkg-1.21.9+nmu1/debian/changelog 2022-09-24 15:56:32.000000000 +0200
@@ -1,3 +1,10 @@
+dpkg (1.21.9+nmu1) UNRELEASED; urgency=medium
+
+ * Non-maintainer upload.
+ * dpkg-genbuildinfo: Emit correct multiarch dependencies. (Closes: #873138)
+
+ -- Helmut Grohne <helmut at subdivi.de> Sat, 24 Sep 2022 15:56:32 +0200
+
dpkg (1.21.9) unstable; urgency=medium
[ Guillem Jover ]
diff --minimal -Nru dpkg-1.21.9/scripts/Dpkg/Deps.pm dpkg-1.21.9+nmu1/scripts/Dpkg/Deps.pm
--- dpkg-1.21.9/scripts/Dpkg/Deps.pm 2022-06-18 19:57:43.000000000 +0200
+++ dpkg-1.21.9+nmu1/scripts/Dpkg/Deps.pm 2022-09-24 15:56:32.000000000 +0200
@@ -421,10 +421,13 @@
my $br = $b->{relation} // 'undef';
my $av = $a->{version} // '';
my $bv = $b->{version} // '';
+ my $aq = $a->{archqual} // '';
+ my $bq = $b->{archqual} // '';
my $res = (($a->{package} cmp $b->{package}) ||
($relation_ordering{$ar} <=> $relation_ordering{$br}) ||
- ($av cmp $bv));
+ ($av cmp $bv) ||
+ ($aq cmp $bq));
return $res if $res != 0;
}
}
diff --minimal -Nru dpkg-1.21.9/scripts/dpkg-genbuildinfo.pl dpkg-1.21.9+nmu1/scripts/dpkg-genbuildinfo.pl
--- dpkg-1.21.9/scripts/dpkg-genbuildinfo.pl 2022-06-30 06:12:21.000000000 +0200
+++ dpkg-1.21.9+nmu1/scripts/dpkg-genbuildinfo.pl 2022-09-24 15:56:32.000000000 +0200
@@ -105,7 +105,7 @@
$facts->add_installed_package($package, $version, $arch, $multiarch);
if (/^Essential: yes$/m) {
- push @essential_pkgs, $package;
+ push @essential_pkgs, "$package:$arch";
}
if (/^Provides: (.*)$/m) {
@@ -157,6 +157,7 @@
sub collect_installed_builddeps {
my $control = shift;
+ my $build_arch = get_build_arch();
my ($facts, $depends, $essential_pkgs) = parse_status("$admindir/status");
my %seen_pkgs;
my @unprocessed_pkgs;
@@ -187,13 +188,16 @@
if ($pkg_name =~ /\A(.*):(.*)\z/) {
$pkg_name = $1;
my $arch = $2;
- $required_architecture = $arch if $arch !~ /\A(?:all|any|native)\Z/
- }
+ $required_architecture = $arch =~ /\A(?:any|native)\Z/ ? $build_arch : $arch;
+ } else {
+ $required_architecture = get_host_arch();
+ }
my $pkg;
my $qualified_pkg_name;
foreach my $installed_pkg (@{$facts->{pkg}->{$pkg_name}}) {
- if (!defined $required_architecture ||
- $required_architecture eq $installed_pkg->{architecture}) {
+ $required_architecture = $installed_pkg->{architecture} if $installed_pkg->{multiarch} eq 'foreign';
+ $required_architecture = 'all' if $required_architecture eq $build_arch and $installed_pkg->{architecture} eq 'all';
+ if ($required_architecture eq $installed_pkg->{architecture}) {
$pkg = $installed_pkg;
$qualified_pkg_name = $pkg_name . ':' . $installed_pkg->{architecture};
last;
@@ -204,7 +208,7 @@
my $architecture = $pkg->{architecture};
my $new_deps_str = defined $depends->{$qualified_pkg_name} ? deps_concat(@{$depends->{$qualified_pkg_name}}) : '';
my $new_deps = deps_parse($new_deps_str);
- if (!defined $required_architecture) {
+ if ($qualified_pkg_name =~ /:(?:all|\Q$build_arch\E)\Z/) {
$installed_deps->add(Dpkg::Deps::Simple->new("$pkg_name (= $version)"));
} else {
$installed_deps->add(Dpkg::Deps::Simple->new("$qualified_pkg_name (= $version)"));
@@ -225,8 +229,11 @@
# We add every sub-dependencies as we cannot know which package
# in an OR dependency has been effectively used.
deps_iterate($new_deps, sub {
+ my $archqual = $required_architecture;
+ $archqual = $build_arch if $archqual eq 'all';
+ $archqual = $_[0]->{archqual} if defined $_[0]->{archqual};
push @unprocessed_pkgs,
- $_[0]->{package} . (defined $_[0]->{archqual} ? ':' . $_[0]->{archqual} : '');
+ $_[0]->{package} . ':' . $archqual;
1
});
} elsif (defined $facts->{virtualpkg}->{$pkg_name}) {
More information about the Reproducible-builds
mailing list