[Qa-jenkins-scm] [Git][qa/jenkins.debian.net][master] update debrebuild fork to latest version from MR 212 from josch for src:debrebuild
Holger Levsen
gitlab at salsa.debian.org
Fri Dec 18 20:36:47 GMT 2020
Holger Levsen pushed to branch master at Debian QA / jenkins.debian.net
Commits:
8e00fda3 by Holger Levsen at 2020-12-18T21:35:47+01:00
update debrebuild fork to latest version from MR 212 from josch for src:debrebuild
commit 4970c8bbf1692f2d69ba00b8a73ffde9659baeab (HEAD -> mr-origin-212)
Author: Johannes 'josch' Schauer <josch at mister-muffin.de>
Date: Thu Dec 3 21:36:46 2020 +0100
debrebuild: perform the actual build using sbuild or mmdebstrap as backends
- Add the --builder option to select the rebuilder backend. The default is
"none" which just prints instructions. Possible values are sbuild, dpkg,
mmdebstrap, sbuild+unshare and pbuilder.
- Add the --output option where the build results will be stored.
- Compare the checksums from the input buildinfo with the build artifacts.
- Unconditionally set Acquire::http::Dl-Limit "1000" and Acquire::Retries
"5" to hopefully not get blocked by snapshot.d.o.
- The sbuild+unshare as well as the mmdebstrap builders allow debrebuild
to operate without any setup. Closes: #958750
- All builders (except the "none" builder) do download the source package
automatically. Closes: #961861
- The "none" builder retains the former behavior of debrebuild. By using
the other builders, debrebuild will actually perform the build itself
and check if the produced build artifacts match. Closes: #955123
- All builders now support binNMUs. Closes: #961862
- Downgrades are avoided by using the last stable release. Closes: #955307
- There is now proper --help output and a man page. Closes: #955049
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
1 changed file:
- bin/rb-debrebuild
Changes:
=====================================
bin/rb-debrebuild
=====================================
@@ -1,6 +1,6 @@
#!/usr/bin/perl
#
-# Copyright © 2014-2016 Johannes Schauer <j.schauer at email.de>
+# Copyright © 2014-2020 Johannes Schauer <j.schauer at email.de>
# Copyright © 2020 Niels Thykier <niels at thykier.net>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
@@ -25,6 +25,7 @@ use Dpkg::Deps;
use Dpkg::Source::Package;
use File::Temp qw(tempdir);
use File::Path qw(make_path);
+use File::HomeDir;
use JSON::PP;
use Time::Piece;
use File::Basename;
@@ -38,20 +39,25 @@ eval {
};
if ($@) {
if ($@ =~ m/Can\'t locate LWP/) {
- die "Unable to run: the libwww-perl package is not installed";
+ die "Unable to run: the libwww-perl package is not installed\n";
} else {
- die "Unable to run: Couldn't load LWP::Simple: $@";
+ die "Unable to run: Couldn't load LWP::Simple: $@\n";
}
}
-my $respect_build_path = 1;
-my $use_tor = 0;
-my $apt_tor_prefix = '';
+my $respect_build_path = 1;
+my $use_tor = 0;
+my $outdir = './';
+my $builder = 'none';
+my @required_timestamps = ();
my %OPTIONS = (
'help|h' => sub { usage(0); },
'use-tor-proxy!' => \$use_tor,
'respect-build-path!' => \$respect_build_path,
+ 'output|O=s' => \$outdir,
+ 'builder=s' => \$builder,
+ 'timestamp|t=s' => \@required_timestamps,
);
sub usage {
@@ -59,7 +65,7 @@ sub usage {
my $me = basename($0);
$exit_code //= 0;
print <<EOF;
-Usage: $me [--no-respect-build-path] <buildinfo>
+Usage: $me [options] <buildinfo>
$me <--help|-h>
Given a buildinfo file from a Debian package, generate instructions for
@@ -72,11 +78,48 @@ Options:
Assumes "apt-transport-tor" is installed both in host + chroot
--[no-]respect-build-path Whether to setup the build to use the Build-Path from the
provided .buildinfo file.
+ --output, -O Directory for the build artifacts (default: ./)
+ --builder=BUILDER Which building software should be used. See section BUILDER
+ --timestamp, -t The required timestamps from snapshot.d.o if you already know them, separated by commas
Note: $me can parse buildinfo files with and without a GPG signature. However,
the signature (if present) is discarded as debrebuild does not support verifying
it. If the authenticity or integrity of the buildinfo files are important to
-you, checking these need to be done before invoking $me.
+you, checking these need to be done before invoking $me, for example by using
+dscverify.
+
+EXAMPLES
+
+ \$ $me --output=./artifacts --builder=mmdebstrap hello_2.10-2_amd64.buildinfo
+
+BUILDERS
+
+debrebuild can use different backends to perform the actual package rebuild.
+The desired backend is chosen using the --builder option. The default is
+"none".
+
+ none Dry-run mode. No build is performed.
+ sbuild Use sbuild to build the package. This requires sbuild to be
+ setup with schroot chroots of Debian stable distributions.
+ mmdebstrap Use mmdebstrap to build the package. This requires no
+ setup and no superuser privileges.
+ dpkg Directly run apt-get and dpkg-buildpackage on the current
+ system without chroot. This requires root privileges..
+ pbuilder Use pbuilder to build the package. This requires pbuilder
+ to be setup with chroots of Debian stable distributions.
+ sbuild+unshare Use sbuild with the unshare backend. This will create the
+ chroot and perform the build without superuser privileges
+ and without any setup.
+
+UNSHARE
+
+The sbuild+unshare builder requires and the mmdebstrap builder benefits from
+having unprivileged user namespaces activated. On Ubuntu they are enabled by
+default but on Debian they are disabled for security reasons. Refer to Debian
+bug #898446 for details. To enable user namespaces, run:
+
+ \$ sudo sysctl -w kernel.unprivileged_userns_clone=1
+
EOF
exit($exit_code);
@@ -84,6 +127,9 @@ EOF
GetOptions(%OPTIONS);
+# support timestamps being separated by a comma
+ at required_timestamps = split(/,/, join(',', @required_timestamps));
+
my $buildinfo = shift @ARGV;
if (not defined($buildinfo)) {
print STDERR "ERROR: Missing mandatory buildinfo filename\n";
@@ -106,8 +152,9 @@ if (@ARGV) {
usage(1);
}
+my $base_mirror = "http://snapshot.debian.org/archive/debian";
if ($use_tor) {
- $apt_tor_prefix = 'tor+';
+ $base_mirror = "tor+http://snapshot.debian.org/archive/debian";
eval {
$LWP::Simple::ua->proxy([qw(http https)] => 'socks://127.0.0.1:9050');
};
@@ -125,7 +172,7 @@ if ($use_tor) {
my $cdata = Dpkg::Control->new(type => CTRL_FILE_BUILDINFO, allow_pgp => 1);
if (not $cdata->load($buildinfo)) {
- die "cannot load $buildinfo";
+ die "cannot load $buildinfo\n";
}
if ($cdata->get_option('is_pgp_signed')) {
@@ -140,21 +187,61 @@ my $build_source = (scalar(grep /^source$/, @architectures)) == 1;
my $build_archall = (scalar(grep /^all$/, @architectures)) == 1;
@architectures = grep { !/^source$/ && !/^all$/ } @architectures;
if (scalar @architectures > 1) {
- die "more than one architecture in Architecture field";
+ die "more than one architecture in Architecture field\n";
}
my $build_archany = (scalar @architectures) == 1;
-my $host_arch = undef;
-if ($build_archany) {
- $host_arch = $architectures[0];
-}
my $build_arch = $cdata->{"Build-Architecture"};
if (not defined($build_arch)) {
- die "need Build-Architecture field";
+ die "need Build-Architecture field\n";
+}
+my $host_arch = $cdata->{"Host-Architecture"};
+if (not defined($host_arch)) {
+ $host_arch = $build_arch;
}
+
+my $srcpkgname = $cdata->{Source};
+my $srcpkgver = $cdata->{Version};
+my $srcpkgbinver
+ = $cdata->{Version}; # this version will include the binmu suffix
+if ($srcpkgname =~ / /) {
+ # In some cases such as binNMUs, the source field contains a version in
+ # the form:
+ # mscgen (0.20)
+ ($srcpkgname, $srcpkgver) = split / /, $srcpkgname, 2;
+ # Add a simple control check to avoid the worst surprises and stop obvious
+ # cases of garbage-in-garbage-out.
+ die("Unexpected source package name: ${srcpkgname}\n")
+ if $srcpkgname =~ m{[ \t_/\(\)<>!\n%&\$\#\@]};
+ # remove the surrounding parenthesis from the version
+ $srcpkgver =~ s/^\((.*)\)$/$1/;
+}
+
+my $new_buildinfo;
+{
+ my $arch;
+ if ($build_archany) {
+ $arch = $host_arch;
+ } elsif ($build_archall) {
+ $arch = 'all';
+ } else {
+ die "nothing to build\n";
+ }
+ $new_buildinfo = "$outdir/${srcpkgname}_${srcpkgbinver}_$arch.buildinfo";
+}
+if (-e $new_buildinfo) {
+ my ($dev1, $ino1) = (lstat $buildinfo)[0, 1]
+ or die "cannot lstat $buildinfo: $!\n";
+ my ($dev2, $ino2) = (lstat $new_buildinfo)[0, 1]
+ or die "cannot lstat $new_buildinfo: $!\n";
+ if ($dev1 == $dev2 && $ino1 == $ino2) {
+ die "refusing to overwrite the input buildinfo file\n";
+ }
+}
+
my $inst_build_deps = $cdata->{"Installed-Build-Depends"};
if (not defined($inst_build_deps)) {
- die "need Installed-Build-Depends field";
+ die "need Installed-Build-Depends field\n";
}
my $custom_build_path = $respect_build_path ? $cdata->{'Build-Path'} : undef;
@@ -184,41 +271,32 @@ qq{Build-Path must be a non-empty absolute path (i.e. start with "/").\n}
}
}
-sub extract_source {
- my ($source) = @_;
- # In some cases such as binNMUs, the source field contains a version in
- # the form:
- # mscgen (0.20)
- # This function strips the version leaving only the source package.
- $source =~ s/\s+\(.+\)\s*//;
- # Add a simple control check to avoid the worst surprises and stop obvious
- # cases of garbage-in-garbage-out.
- die("Unexpected source package name: ${source}\n")
- if $source =~ m{[ \t_/\(\)<>!\n%&\$\#\@]};
- return $source;
-}
-
my $srcpkg = Dpkg::Source::Package->new();
-$srcpkg->{fields}{'Source'} = extract_source($cdata->{"Source"});
-$srcpkg->{fields}{'Version'} = $cdata->{"Version"};
+$srcpkg->{fields}{'Source'} = $srcpkgname;
+$srcpkg->{fields}{'Version'} = $srcpkgver;
my $dsc_fname
= (dirname($buildinfo)) . '/' . $srcpkg->get_basename(1) . ".dsc";
my $environment = $cdata->{"Environment"};
if (not defined($environment)) {
- die "need Environment field";
+ die "need Environment field\n";
}
$environment =~ s/\n/ /g; # remove newlines
$environment =~ s/^ //; # remove leading whitespace
-my $checksums = Dpkg::Checksums->new();
-$checksums->add_from_control($cdata);
-my @files = $checksums->get_files();
+my @environment;
+foreach my $line (split /\n/, $cdata->{"Environment"}) {
+ chomp $line;
+ if ($line eq '') {
+ next;
+ }
+ my ($name, $val) = split /=/, $line, 2;
+ $val =~ s/^"(.*)"$/$1/;
+ push @environment, "$name=$val";
+}
# gather all installed build-depends and figure out the version of base-files
-# and dpkg
my $base_files_version;
-my $dpkg_version;
my @inst_build_deps = ();
$inst_build_deps
= deps_parse($inst_build_deps, reduce_arch => 0, build_dep => 0);
@@ -231,24 +309,18 @@ foreach my $pkg ($inst_build_deps->get_deps()) {
die "dependency disjunctions are not allowed\n";
}
if (not defined($pkg->{package})) {
- die "name undefined";
+ die "name undefined\n";
}
if (defined($pkg->{relation})) {
if ($pkg->{relation} ne "=") {
die "wrong relation";
}
if (not defined($pkg->{version})) {
- die "version undefined";
+ die "version undefined\n";
}
} else {
die "no version";
}
- if ($pkg->{package} eq "dpkg") {
- if (defined($dpkg_version)) {
- die "more than one dpkg\n";
- }
- $dpkg_version = $pkg->{version};
- }
if ($pkg->{package} eq "base-files") {
if (defined($base_files_version)) {
die "more than one base-files\n";
@@ -265,53 +337,102 @@ foreach my $pkg ($inst_build_deps->get_deps()) {
if (!defined($base_files_version)) {
die "no base-files\n";
}
-if (!defined($dpkg_version)) {
- die "no dpkg\n";
-}
-# figure out the debian release from the version of base-files and dpkg
-# FIXME - this really shouldn't be hardcoded here
+# figure out the debian release from the version of base-files
my $base_dist;
-my %base_files_map = (
- "6" => "squeeze",
- "7" => "wheezy",
- "8" => "jessie",
- "9" => "stretch",
- "10" => "buster",
- "11" => "bullseye",
- "12" => "bookworm",
-);
-my %dpkg_map = (
- "15" => "squeeze",
- "16" => "wheezy",
- "17" => "jessie",
- "18" => "stretch",
- "19" => "buster",
- "20" => "bullseye",
- "21" => "bookworm",
-);
+my %base_files_map = ();
+my $di_path = '/usr/share/distro-info/debian.csv';
+eval { require Debian::DistroInfo; };
+if (!$@) {
+ # libdistro-info-perl is installed
+ my $di = DebianDistroInfo->new();
+ foreach my $series ($di->all) {
+ if (!$di->version($series)) {
+ next;
+ }
+ $base_files_map{ $di->version($series) } = $series;
+ }
+} elsif (-f $di_path) {
+ # distro-info-data is installed
+ open my $fh, '<', $di_path or die "cannot open $di_path: $!\n";
+ my $i = 0;
+ while (my $line = <$fh>) {
+ chomp($line);
+ $i++;
+ my @cells = split /,/, $line;
+ if (scalar @cells < 4) {
+ die "cannot parse line $i of $di_path\n";
+ }
+ if (
+ $i == 1
+ and ( scalar @cells < 6
+ or $cells[0] ne 'version'
+ or $cells[1] ne 'codename'
+ or $cells[2] ne 'series'
+ or $cells[3] ne 'created'
+ or $cells[4] ne 'release'
+ or $cells[5] ne 'eol')
+ ) {
+ die "cannot find correct header in $di_path\n";
+ }
+ if ($i == 1) {
+ next;
+ }
+ $base_files_map{ $cells[0] } = $cells[2];
+ }
+ close $fh;
+} else {
+ # nothing is installed -- use hard-coded values
+ %base_files_map = (
+ "6" => "squeeze",
+ "7" => "wheezy",
+ "8" => "jessie",
+ "9" => "stretch",
+ "10" => "buster",
+ "11" => "bullseye",
+ "12" => "bookworm",
+ );
+}
$base_files_version =~ s/^(\d+).*/$1/;
-#$dpkg_version =~ s/1\.(\d+)\..*/$1/;
-$base_dist = $base_files_map{$base_files_version};
+# we subtract one from $base_files_version because we want the Debian release
+# before what is currently in unstable
+$base_dist = $base_files_map{ $base_files_version - 1 };
if (!defined $base_dist) {
- die "base-files version didn't map to any Debian release";
+ die "base-files version didn't map to any Debian release\n";
}
-#if ($base_dist ne $dpkg_map{$dpkg_version}) {
-# die "base-files and dpkg versions point to different Debian releases\n";
-#}
-
-# test if all checksums in the buildinfo file check out
-
-#foreach my $fname ($checksums->get_files()) {
-# # Re-adding existing files to the checksum object is the current way to
-# # ask Dpkg to check the checksums for us
-# $checksums->add_from_file((dirname($buildinfo)) . '/' . $fname);
-#}
+my $src_date;
+{
+ print "retrieving snapshot.d.o data for $srcpkgname $srcpkgver\n";
+ my $json_url
+ = "http://snapshot.debian.org/mr/package/$srcpkgname/$srcpkgver/srcfiles?fileinfo=1";
+ my $content = LWP::Simple::get($json_url);
+ die "cannot retrieve $json_url" unless defined $content;
+ my $json = JSON::PP->new();
+ # json options taken from debsnap
+ my $json_text = $json->allow_nonref->utf8->relaxed->decode($content);
+ die "cannot decode json" unless defined $json_text;
+ foreach my $result (@{ $json_text->{result} }) {
+ # FIXME - assumption: package is from Debian official (and not ports)
+ my @package_from_main = grep { $_->{archive_name} eq "debian" }
+ @{ $json_text->{fileinfo}->{ $result->{hash} } };
+ if (scalar @package_from_main > 1) {
+ die
+ "more than one package with the same hash in Debian official\n";
+ }
+ if (scalar @package_from_main == 0) {
+ die "no package with the right hash in Debian official\n";
+ }
+ $src_date = $package_from_main[0]->{first_seen};
+ }
+}
+if (!defined($src_date)) {
+ die "cannot find .dsc\n";
+}
# setup a temporary apt directory
@@ -327,10 +448,32 @@ foreach my $d ((
make_path("$tempdir/$d");
}
+# We use the Build-Date field as a heuristic to find a good date for the
+# stable release. If we would get the stable release from deb.debian.org
+# instead, then packages might be newer than in unstable of the past because
+# of point releases. The date from the source package will also work in most
+# cases but will fail for binNMU buildinfo files where the source package
+# might even come from years in the past
+my $build_date;
+{
+ local $ENV{LC_ALL} = 'C';
+ my $tp
+ = Time::Piece->strptime($cdata->{'Build-Date'}, '%a, %d %b %Y %T %z');
+ $build_date = $tp->strftime("%Y%m%dT%H%M%SZ");
+}
+
+sub get_sources_list() {
+ my @result = ();
+ push @result, "deb $base_mirror/$build_date/ $base_dist main";
+ push @result, "deb-src $base_mirror/$src_date/ unstable main";
+ foreach my $ts (@required_timestamps) {
+ push @result, "deb $base_mirror/$ts/ unstable main";
+ }
+ return @result;
+}
+
open(FH, '>', "$tempdir/etc/apt/sources.list");
-print FH <<EOF;
-deb ${apt_tor_prefix}http://deb.debian.org/debian/ $base_dist main
-EOF
+print FH (join "\n", get_sources_list) . "\n";
close FH;
# FIXME - document what's dpkg's status for
# Create dpkg status
@@ -382,6 +525,9 @@ Dir "$tempdir";
Dir::State::status "$tempdir/var/lib/dpkg/status";
Acquire::Check-Valid-Until "false";
Acquire::Languages "none";
+Acquire::http::Dl-Limit "1000";
+Acquire::https::Dl-Limit "1000";
+Acquire::Retries "5";
Binary::apt-get::Acquire::AllowInsecureRepositories "false";
EOF
close FH;
@@ -434,272 +580,621 @@ sub parse_all_packages_files {
}
my $index = parse_all_packages_files();
+if (scalar @required_timestamps == 0) {
+ # go through all packages in the Installed-Build-Depends field and find out
+ # the timestamps at which they were first seen each
+ my %notfound_timestamps;
-# go through all packages in the Installed-Build-Depends field and find out
-# the timestamps at which they were first seen each
-my %notfound_timestamps;
+ my %missing;
-my (%missing, $snapshots_needed);
+ foreach my $pkg (@inst_build_deps) {
+ my $pkg_name = $pkg->{name};
+ my $pkg_ver = $pkg->{version};
+ my $pkg_arch = $pkg->{architecture};
+
+ # check if we really need to acquire this package from snapshot.d.o or if
+ # it already exists in the cache
+ if (defined $pkg->{architecture}) {
+ if ($index->get_by_key("$pkg_name $pkg_ver $pkg_arch")) {
+ print "skipping $pkg_name $pkg_ver\n";
+ next;
+ }
+ } else {
+ if ($index->get_by_key("$pkg_name $pkg_ver $build_arch")) {
+ $pkg->{architecture} = $build_arch;
+ print "skipping $pkg_name $pkg_ver\n";
+ next;
+ }
+ if ($index->get_by_key("$pkg_name $pkg_ver all")) {
+ $pkg->{architecture} = "all";
+ print "skipping $pkg_name $pkg_ver\n";
+ next;
+ }
+ }
-foreach my $pkg (@inst_build_deps) {
- my $pkg_name = $pkg->{name};
- my $pkg_ver = $pkg->{version};
- my $pkg_arch = $pkg->{architecture};
+ print "retrieving snapshot.d.o data for $pkg_name $pkg_ver\n";
+ my $json_url
+ = "http://snapshot.debian.org/mr/binary/$pkg_name/$pkg_ver/binfiles?fileinfo=1";
+ my $content = LWP::Simple::get($json_url);
+ die "cannot retrieve $json_url" unless defined $content;
+ my $json = JSON::PP->new();
+ # json options taken from debsnap
+ my $json_text = $json->allow_nonref->utf8->relaxed->decode($content);
+ die "cannot decode json" unless defined $json_text;
+ my $pkg_hash;
+ if (scalar @{ $json_text->{result} } == 1) {
+ # if there is only a single result, then the package must either be
+ # Architecture:all, be the build architecture or match the requested
+ # architecture
+ $pkg_hash = ${ $json_text->{result} }[0]->{hash};
+ $pkg->{architecture}
+ = ${ $json_text->{result} }[0]->{architecture};
+ # if a specific architecture was requested, it should match
+ if (defined $pkg_arch && $pkg_arch ne $pkg->{architecture}) {
+ die
+"package $pkg_name was explicitly requested for $pkg_arch but only $pkg->{architecture} was found\n";
+ }
+ # if no specific architecture was requested, it should be the build
+ # architecture
+ if ( !defined $pkg_arch
+ && $build_arch ne $pkg->{architecture}
+ && "all" ne $pkg->{architecture}) {
+ die
+"package $pkg_name was implicitly requested for $pkg_arch but only $pkg->{architecture} was found\n";
+ }
+ # Ensure that $pkg_arch is defined from here as we want to look it up
+ # later in a Packages file from snapshot.d.o if it is not in the
+ # current Packages file
+ $pkg_arch = $pkg->{architecture};
+ } else {
+ # Since the package occurs more than once, we expect it to be of
+ # Architecture:any
+ #
+ # If no specific architecture was requested, look for the build
+ # architecture
+ if (!defined $pkg_arch) {
+ $pkg_arch = $build_arch;
+ }
+ foreach my $result (@{ $json_text->{result} }) {
+ if ($result->{architecture} eq $pkg_arch) {
+ $pkg_hash = $result->{hash};
+ last;
+ }
+ }
+ if (!defined($pkg_hash)) {
+ die "cannot find package in architecture $pkg_arch\n";
+ }
+ # we now know that this package is not architecture:all but has a
+ # concrete architecture
+ $pkg->{architecture} = $pkg_arch;
+ }
+ # FIXME - assumption: package is from Debian official (and not ports)
+ my @package_from_main = grep { $_->{archive_name} eq "debian" }
+ @{ $json_text->{fileinfo}->{$pkg_hash} };
+ if (scalar @package_from_main > 1) {
+ die
+ "more than one package with the same hash in Debian official\n";
+ }
+ if (scalar @package_from_main == 0) {
+ die "no package with the right hash in Debian official\n";
+ }
+ my $date = $package_from_main[0]->{first_seen};
+ $pkg->{first_seen} = $date;
+ $notfound_timestamps{$date} = 1;
+ $missing{"${pkg_name}/${pkg_ver}/${pkg_arch}"} = 1;
+ }
- # check if we really need to acquire this package from snapshot.d.o or if
- # it already exists in the cache
- if (defined $pkg->{architecture}) {
- if ($index->get_by_key("$pkg_name $pkg_ver $pkg_arch")) {
- print "skipping $pkg_name $pkg_ver\n";
+ # feed apt with timestamped snapshot.debian.org URLs until apt is able to
+ # find all the required package versions. We start with the most recent
+ # timestamp, check which packages cannot be found at that timestamp, add
+ # the timestamp of the most recent not-found package and continue doing
+ # this iteratively until all versions can be found.
+
+ while (0 < scalar keys %notfound_timestamps) {
+ print "left to check: " . (scalar keys %notfound_timestamps) . "\n";
+ my @timestamps = map { Time::Piece->strptime($_, '%Y%m%dT%H%M%SZ') }
+ (sort keys %notfound_timestamps);
+ my $newest = $timestamps[$#timestamps];
+ $newest = $newest->strftime("%Y%m%dT%H%M%SZ");
+ push @required_timestamps, $newest;
+ delete $notfound_timestamps{$newest};
+
+ my $snapshot_url = "$base_mirror/$newest/";
+
+ open(FH, '>>', "$tempdir/etc/apt/sources.list");
+ print FH "deb ${snapshot_url} unstable main\n";
+ close FH;
+
+ 0 == system 'apt-get', 'update' or die "apt-get update failed\n";
+
+ my $index = parse_all_packages_files();
+ foreach my $pkg (@inst_build_deps) {
+ my $pkg_name = $pkg->{name};
+ my $pkg_ver = $pkg->{version};
+ my $pkg_arch = $pkg->{architecture};
+ my $first_seen = $pkg->{first_seen};
+ my $cdata = $index->get_by_key("$pkg_name $pkg_ver $pkg_arch");
+ if (not defined($cdata->{"Package"})) {
+ # Not present yet; we hope a later snapshot URL will locate it.
+ next;
+ }
+ delete($missing{"${pkg_name}/${pkg_ver}/${pkg_arch}"});
+ if (defined $first_seen) {
+ # this may delete timestamps that we actually need for some other
+ # packages
+ delete $notfound_timestamps{$first_seen};
+ }
+ }
+ }
+
+ if (%missing) {
+ print STDERR 'Cannot locate the following packages via snapshots'
+ . " or the current repo/mirror\n";
+ for my $key (sort(keys(%missing))) {
+ print STDERR " ${key}\n";
+ }
+ exit(1);
+ }
+} else {
+ # find out the actual package architecture for all installed build
+ # dependencies without explicit architecture qualification
+ foreach my $pkg (@inst_build_deps) {
+ my $pkg_name = $pkg->{name};
+ my $pkg_ver = $pkg->{version};
+ if (defined $pkg->{architecture}) {
next;
}
- } else {
if ($index->get_by_key("$pkg_name $pkg_ver $build_arch")) {
$pkg->{architecture} = $build_arch;
- print "skipping $pkg_name $pkg_ver\n";
next;
}
if ($index->get_by_key("$pkg_name $pkg_ver all")) {
$pkg->{architecture} = "all";
- print "skipping $pkg_name $pkg_ver\n";
next;
}
+ die "cannot find $pkg_name $pkg_ver in index\n";
}
+}
- print "retrieving snapshot.d.o data for $pkg_name $pkg_ver\n";
- my $json_url
- = "http://snapshot.debian.org/mr/binary/$pkg_name/$pkg_ver/binfiles?fileinfo=1";
- my $content = LWP::Simple::get($json_url);
- die "cannot retrieve $json_url" unless defined $content;
- my $json = JSON::PP->new();
- # json options taken from debsnap
- my $json_text = $json->allow_nonref->utf8->relaxed->decode($content);
- die "cannot decode json" unless defined $json_text;
- my $pkg_hash;
- if (scalar @{ $json_text->{result} } == 1) {
- # if there is only a single result, then the package must either be
- # Architecture:all, be the build architecture or match the requested
- # architecture
- $pkg_hash = ${ $json_text->{result} }[0]->{hash};
- $pkg->{architecture} = ${ $json_text->{result} }[0]->{architecture};
- # if a specific architecture was requested, it should match
- if (defined $pkg_arch && $pkg_arch ne $pkg->{architecture}) {
- die
-"package $pkg_name was explicitly requested for $pkg_arch but only $pkg->{architecture} was found\n";
- }
- # if no specific architecture was requested, it should be the build
- # architecture
- if ( !defined $pkg_arch
- && $build_arch ne $pkg->{architecture}
- && "all" ne $pkg->{architecture}) {
- die
-"package $pkg_name was implicitly requested for $pkg_arch but only $pkg->{architecture} was found\n";
- }
- # Ensure that $pkg_arch is defined from here as we want to look it up
- # later in a Packages file from snapshot.d.o if it is not in the
- # current Packages file
- $pkg_arch = $pkg->{architecture};
- } else {
- # Since the package occurs more than once, we expect it to be of
- # Architecture:any
- #
- # If no specific architecture was requested, look for the build
- # architecture
- if (!defined $pkg_arch) {
- $pkg_arch = $build_arch;
- }
- foreach my $result (@{ $json_text->{result} }) {
- if ($result->{architecture} eq $pkg_arch) {
- $pkg_hash = $result->{hash};
- last;
- }
- }
- if (!defined($pkg_hash)) {
- die "cannot find package in architecture $pkg_arch\n";
- }
- # we now know that this package is not architecture:all but has a
- # concrete architecture
- $pkg->{architecture} = $pkg_arch;
- }
- # FIXME - assumption: package is from Debian official (and not ports)
- my @package_from_main = grep { $_->{archive_name} eq "debian" }
- @{ $json_text->{fileinfo}->{$pkg_hash} };
- if (scalar @package_from_main > 1) {
- die "more than one package with the same hash in Debian official\n";
- }
- if (scalar @package_from_main == 0) {
- die "no package with the right hash in Debian official\n";
- }
- my $date = $package_from_main[0]->{first_seen};
- $pkg->{first_seen} = $date;
- $notfound_timestamps{$date} = 1;
- $missing{"${pkg_name}/${pkg_ver}/${pkg_arch}"} = 1;
-}
-
-# feed apt with timestamped snapshot.debian.org URLs until apt is able to find
-# all the required package versions. We start with the most recent timestamp,
-# check which packages cannot be found at that timestamp, add the timestamp of
-# the most recent not-found package and continue doing this iteratively until
-# all versions can be found.
-
-while (0 < scalar keys %notfound_timestamps) {
- print "left to check: " . (scalar keys %notfound_timestamps) . "\n";
- my @timestamps = map { Time::Piece->strptime($_, '%Y%m%dT%H%M%SZ') }
- (sort keys %notfound_timestamps);
- my $newest = $timestamps[$#timestamps];
- $newest = $newest->strftime("%Y%m%dT%H%M%SZ");
- delete $notfound_timestamps{$newest};
-
- my $snapshot_url = "http://snapshot.debian.org/archive/debian/$newest/";
-
- open(FH, '>>', "$tempdir/etc/apt/sources.list");
- print FH "deb ${apt_tor_prefix}${snapshot_url} unstable main\n";
- close FH;
+# remove $tempdir manually to avoid any surprises
+0 == system 'apt-get', '--option',
+ 'Dir::Etc::SourceList=/dev/null', '--option',
+ 'Dir::Etc::SourceParts=/dev/null', 'update'
+ or die "apt-get update failed\n";
+
+foreach my $f (
+ '/var/cache/apt/pkgcache.bin',
+ '/var/cache/apt/srcpkgcache.bin',
+ '/var/lib/dpkg/status',
+ '/var/lib/apt/lists/lock',
+ '/etc/apt/apt.conf',
+ '/etc/apt/sources.list',
+ '/etc/apt/trusted.gpg.d/debian-archive-removed-keys.gpg',
+ '/etc/apt/trusted.gpg.d/debian-archive-keyring.gpg'
+) {
+ unlink "$tempdir/$f" or die "cannot unlink $f: $!\n";
+}
- 0 == system 'apt-get', 'update' or die "apt-get update failed";
+foreach my $d (
+ '/var/cache/apt/archives/partial', '/var/cache/apt/archives',
+ '/var/cache/apt', '/var/cache',
+ '/var/lib/dpkg', '/var/lib/apt/lists/auxfiles',
+ '/var/lib/apt/lists/partial', '/var/lib/apt/lists',
+ '/var/lib/apt', '/var/lib',
+ '/var', '/etc/apt/sources.list.d',
+ '/etc/apt/trusted.gpg.d', '/etc/apt/preferences.d',
+ '/etc/apt/apt.conf.d', '/etc/apt',
+ '/etc', ''
+) {
+ rmdir "$tempdir/$d" or die "cannot rmdir $d: $!\n";
+}
- my $index = parse_all_packages_files();
- foreach my $pkg (@inst_build_deps) {
- my $pkg_name = $pkg->{name};
- my $pkg_ver = $pkg->{version};
- my $pkg_arch = $pkg->{architecture};
- my $first_seen = $pkg->{first_seen};
- my $cdata = $index->get_by_key("$pkg_name $pkg_ver $pkg_arch");
- if (not defined($cdata->{"Package"})) {
- # Not present yet; we hope a later snapshot URL will locate it.
- next;
- }
- $snapshots_needed = 1;
- delete($missing{"${pkg_name}/${pkg_ver}/${pkg_arch}"});
- if (defined $first_seen) {
- delete $notfound_timestamps{$first_seen};
- }
+!-e $tempdir or die "failed to remove $tempdir\n";
+
+# avoid dependency on String::ShellQuote by implementing the mechanism
+# from python's shlex.quote function
+sub shellescape ($) {
+ my $string = shift;
+ if (length $string == 0) {
+ return "''";
+ }
+ # search for occurrences of characters that are not safe
+ # the 'a' regex modifier makes sure that \w only matches ASCII
+ if ($string !~ m/[^\w@\%+=:,.\/-]/a) {
+ return $string;
}
+ # wrap the string in single quotes and handle existing single quotes by
+ # putting them outside of the single-quoted string
+ $string =~ s/'/'"'"'/g;
+ return "'$string'";
}
-if (%missing) {
- print STDERR 'Cannot locate the following packages via snapshots'
- . " or the current repo/mirror\n";
- for my $key (sort(keys(%missing))) {
- print STDERR " ${key}\n";
+if ($builder ne "none") {
+ if (!-e $outdir) {
+ make_path($outdir);
}
- exit(1);
}
-print "\n";
-print "Manual installation and build\n";
-print "-----------------------------\n";
-print "\n";
-if ($cdata->{"Binary-Only-Changes"}) {
- print
+my @aptopts = (
+ 'Acquire::Check-Valid-Until "false";',
+ 'Acquire::http::Dl-Limit "1000";',
+ 'Acquire::https::Dl-Limit "1000";',
+ 'Acquire::Retries "5";'
+);
+
+if ($builder eq "none") {
+ print "\n";
+ print "Manual installation and build\n";
+ print "-----------------------------\n";
+ print "\n";
+ if ($cdata->{"Binary-Only-Changes"}) {
+ print
"The buildinfo appears to be for a binNMU; this is not fully supported yet.\n\n";
-}
-print "The following sources.list contains all the required repositories:\n";
-print "\n";
-0 == system 'cat', "$tempdir/etc/apt/sources.list"
- or die "cannot cat $tempdir/etc/apt/sources.list";
-print "\n";
-print "You can manually install the right dependencies like this:\n";
-print "\n";
-print "apt-get install --no-install-recommends";
+ }
+ print
+ "The following sources.list contains all the required repositories:\n";
+ print "\n";
+ print(join "\n", get_sources_list);
+ print "\n";
+ print "You can manually install the right dependencies like this:\n";
+ print "\n";
+ print "apt-get install --no-install-recommends";
-if ($snapshots_needed) {
# Release files from snapshots.d.o have often expired by the time
# we fetch them. Include the option to work around that to assist
# the user.
print " -oAcquire::Check-Valid-Until=false";
-}
-foreach my $pkg (@inst_build_deps) {
- my $pkg_name = $pkg->{name};
- my $pkg_ver = $pkg->{version};
- my $pkg_arch = $pkg->{architecture};
- if ($pkg_arch eq "all" || $pkg_arch eq $build_arch) {
- print " $pkg_name=$pkg_ver";
+ foreach my $pkg (@inst_build_deps) {
+ my $pkg_name = $pkg->{name};
+ my $pkg_ver = $pkg->{version};
+ my $pkg_arch = $pkg->{architecture};
+ if ($pkg_arch eq "all" || $pkg_arch eq $build_arch) {
+ print " $pkg_name=$pkg_ver";
+ } else {
+ print " $pkg_name:$pkg_arch=$pkg_ver";
+ }
+ }
+ print "\n";
+ print "\n";
+ print "And then build your package:\n";
+ print "\n";
+ if ($custom_build_path) {
+ require Cwd;
+ my $custom_build_parent_dir = dirname($custom_build_path);
+ my $dsc_path = Cwd::realpath($dsc_fname)
+ // die("Cannot resolve ${dsc_fname}: $!\n");
+ print "mkdir -p \"${custom_build_parent_dir}\"\n";
+ print qq{dpkg-source -x "${dsc_path}" "${custom_build_path}"\n};
+ print "cd \"$custom_build_path\"\n";
} else {
- print " $pkg_name:$pkg_arch=$pkg_ver";
- }
-}
-print "\n";
-print "\n";
-print "And then build your package:\n";
-print "\n";
-if ($custom_build_path) {
- require Cwd;
- my $custom_build_parent_dir = dirname($custom_build_path);
- my $dsc_path = Cwd::realpath($dsc_fname)
- // die("Cannot resolve ${dsc_fname}: $!\n");
- print "mkdir -p \"${custom_build_parent_dir}\"\n";
- print qq{dpkg-source -x "${dsc_path}" "${custom_build_path}"\n};
- print "cd \"$custom_build_path\"\n";
-} else {
- print qq{dpkg-source -x "${dsc_fname}"\n};
- print "cd packagedirectory\n";
-}
-print "$environment dpkg-buildpackage\n";
-print "\n";
-print "Using sbuild\n";
-print "------------\n";
-print "\n";
+ print qq{dpkg-source -x "${dsc_fname}"\n};
+ print "cd packagedirectory\n";
+ }
+ print "$environment dpkg-buildpackage\n";
+} elsif ($builder eq "dpkg") {
+ if ("$build_arch\n" ne `dpkg --print-architecture`) {
+ die "must be run on $build_arch\n";
+ }
-if ($cdata->{"Binary-Only-Changes"}) {
- print
-"The buildinfo appears to be for a binNMU; this is not fully supported yet.\n\n";
-}
+ if ($> != 0) {
+ die "you must be root for the dpkg builder\n";
+ }
-print "You can try to build the package with sbuild like this:\n";
-print "\n";
-print "SBUILD_CMDLINE=$environment sbuild";
-open(FH, '<', "$tempdir/etc/apt/sources.list");
+ if (-e $custom_build_path) {
+ die "$custom_build_path exists -- refusing to overwrite\n";
+ }
+
+ my $sources = '/etc/apt/sources.list.d/debrebuild.list';
+ if (-e $sources) {
+ die "$sources already exists -- refusing to overwrite\n";
+ }
+ open(FH, '>', $sources) or die "cannot open $sources: $!\n";
+ print FH (join "\n", get_sources_list) . "\n";
+ close FH;
+
+ my $config = '/etc/apt/apt.conf.d/23-debrebuild.conf';
+ if (-e $config) {
+ die "$config already exists -- refusing to overwrite\n";
+ }
+ open(FH, '>', $config) or die "cannot open $config: $!\n";
+ foreach my $line (@aptopts) {
+ print FH "$line\n";
+ }
+ close FH;
+
+ 0 == system 'apt-get', 'update' or die "apt-get update failed\n";
+
+ my @cmd = ('apt-get', 'install', '--no-install-recommends', '--yes');
+ foreach my $pkg (@inst_build_deps) {
+ my $pkg_name = $pkg->{name};
+ my $pkg_ver = $pkg->{version};
+ my $pkg_arch = $pkg->{architecture};
+ if ($pkg_arch eq "all" || $pkg_arch eq $build_arch) {
+ push @cmd, "$pkg_name=$pkg_ver";
+ } else {
+ push @cmd, "$pkg_name:$pkg_arch=$pkg_ver";
+ }
+ }
+ 0 == system @cmd or die "apt-get install failed\n";
+
+ 0 == system 'apt-get', 'source', '--only-source', '--download-only',
+ "$srcpkgname=$srcpkgver"
+ or die "apt-get source failed\n";
+ unlink $sources or die "failed to unlink $sources\n";
+ unlink $config or die "failed to unlink $config\n";
+ make_path(dirname $custom_build_path);
+ 0 == system 'dpkg-source', '--no-check', '--extract',
+ $srcpkg->get_basename(1) . '.dsc', $custom_build_path
+ or die "dpkg-source failed\n";
+ if ($cdata->{"Binary-Only-Changes"}) {
+ open my $infh, '<', "$custom_build_path/debian/changelog"
+ or die "cannot open debian/changelog for reading: $!\n";
+ my $changelogcontent = do { local $/; <$infh> };
+ close $infh;
+ open my $outfh, '>', "$custom_build_path/debian/changelog"
+ or die "cannot open debian/changelog for writing: $!\n";
+ my $logentry = $cdata->{"Binary-Only-Changes"};
+ # due to storing the binnmu changelog entry in deb822 buildinfo, the
+ # first character is an unwanted newline
+ $logentry =~ s/^\n//;
+ print $outfh $logentry;
+ # while the linebreak at the beginning is wrong, there are two missing
+ # at the end
+ print $outfh "\n\n";
+ print $outfh $changelogcontent;
+ close $outfh;
+ }
+ my $build = '';
+ my $changesarch = '';
+ if ($build_archany and $build_archall) {
+ $build = "binary";
+ $changesarch = $host_arch;
+ } elsif ($build_archany and !$build_archall) {
+ $build = "any";
+ $changesarch = $host_arch;
+ } elsif (!$build_archany and $build_archall) {
+ $build = "all";
+ $changesarch = 'all';
+ } else {
+ die "nothing to build\n";
+ }
+ 0 == system 'env', "--chdir=$custom_build_path", @environment,
+ 'dpkg-buildpackage', '-uc', "--host-arch=$host_arch", "--build=$build"
+ or die "dpkg-buildpackage failed\n";
+ # we are not interested in the unpacked source directory
+ 0 == system 'rm', '-r', $custom_build_path;
+ # but instead we want the produced artifacts
+ 0 == system 'dcmd', 'mv',
+ (dirname $custom_build_path)
+ . "/${srcpkgname}_${srcpkgbinver}_$changesarch.changes", $outdir
+ or die "dcmd failed\n";
+} elsif ($builder eq "sbuild" or $builder eq "sbuild+unshare") {
+ my $tarballpath = File::HomeDir->my_home
+ . "/.cache/sbuild/$base_dist-$build_arch.tar.gz";
+ if ($builder eq "sbuild+unshare") {
+ if (!-e $tarballpath) {
+ my $chrootdir = tempdir();
+ 0 == system 'sbuild-createchroot', '--chroot-mode=unshare',
+ '--make-sbuild-tarball', $tarballpath,
+ $base_dist, $chrootdir, "$base_mirror/$build_date/"
+ or die "sbuild-createchroot failed\n";
+ !-e $chrootdir or die "$chrootdir wasn't removed\n";
+ }
+ }
+
+ my @cmd = ('env', "--chdir=$outdir", @environment, 'sbuild');
+ foreach my $line (get_sources_list) {
+ push @cmd, "--extra-repository=$line";
+ }
-while (my $line = <FH>) {
- chomp $line;
- print " --extra-repository=\"$line\"";
-}
-close FH;
-if ($snapshots_needed) {
# Release files from snapshots.d.o have often expired by the time
# we fetch them. Include the option to work around that to assist
# the user.
- print
-q{ --chroot-setup-commands='echo "Acquire::Check-Valid-Until \"false\";" | tee /etc/apt/apt.conf.d/23-debrebuild.conf'};
-}
-my @add_depends = ();
-foreach my $pkg (@inst_build_deps) {
- my $pkg_name = $pkg->{name};
- my $pkg_ver = $pkg->{version};
- my $pkg_arch = $pkg->{architecture};
- if ($pkg_arch eq "all" || $pkg_arch eq $build_arch) {
- push @add_depends, "$pkg_name (= $pkg_ver)";
+ push @cmd,
+ '--chroot-setup-commands=echo '
+ . (shellescape(join '\n', @aptopts))
+ . ' | tee /etc/apt/apt.conf.d/23-debrebuild.conf';
+
+ my @add_depends = ();
+ foreach my $pkg (@inst_build_deps) {
+ my $pkg_name = $pkg->{name};
+ my $pkg_ver = $pkg->{version};
+ my $pkg_arch = $pkg->{architecture};
+ if ($pkg_arch eq "all" || $pkg_arch eq $build_arch) {
+ push @add_depends, "$pkg_name (= $pkg_ver)";
+ } else {
+ push @add_depends, "$pkg_name:$pkg_arch (= $pkg_ver)";
+ }
+ }
+ push @cmd, "--add-depends=" . (join ",", @add_depends);
+ push @cmd, "--build=$build_arch";
+ push @cmd, "--host=$host_arch";
+
+ if ($build_source) {
+ push @cmd, '--source';
} else {
- push @add_depends, "$pkg_name:$pkg_arch (= $pkg_ver)";
+ push @cmd, '--no-source';
}
-}
-print " --add-depends=\"" . (join ",", @add_depends) . "\"";
-print " --build-dep-resolver=aptitude";
-print " --build=$build_arch";
-if (defined $host_arch) {
- print " --host=$host_arch";
-}
-if ($build_source) {
- print " --source";
-} else {
- print " --no-source";
-}
-if ($build_archany) {
- print " --arch-any";
-} else {
- print " --no-arch-any";
-}
-if ($build_archall) {
- print " --arch-all";
+ if ($build_archany) {
+ push @cmd, '--arch-any';
+ } else {
+ push @cmd, '--no-arch-any';
+ }
+ if ($build_archall) {
+ push @cmd, '--arch-all';
+ } else {
+ push @cmd, '--no-arch-all';
+ }
+ if ($cdata->{"Binary-Only-Changes"}) {
+ push @cmd, "--binNMU-changelog=$cdata->{'Binary-Only-Changes'}";
+ }
+ if ($builder eq "sbuild+unshare") {
+ push @cmd, "--chroot=$tarballpath";
+ push @cmd, "--chroot-mode=unshare";
+ }
+ push @cmd, "--dist=$base_dist";
+ push @cmd, "--no-run-lintian";
+ push @cmd, "--no-run-autopkgtest";
+ push @cmd, "--no-apt-upgrade";
+ push @cmd, "--no-apt-distupgrade";
+ if ($custom_build_path) {
+ push @cmd, "--build-path=$custom_build_path";
+ }
+ push @cmd, "${srcpkgname}_$srcpkgver";
+ print(join " ", @cmd) . "\n";
+ 0 == system @cmd or die "sbuild failed\n";
+} elsif ($builder eq "mmdebstrap") {
+
+ my @install = ();
+ foreach my $pkg (@inst_build_deps) {
+ my $pkg_name = $pkg->{name};
+ my $pkg_ver = $pkg->{version};
+ my $pkg_arch = $pkg->{architecture};
+ if ($pkg_arch eq "all" || $pkg_arch eq $build_arch) {
+ push @install, "$pkg_name=$pkg_ver";
+ } else {
+ push @install, "$pkg_name:$pkg_arch=$pkg_ver";
+ }
+ }
+
+ my @binnmucmds = ();
+ if ($cdata->{"Binary-Only-Changes"}) {
+ my $logentry = $cdata->{"Binary-Only-Changes"};
+ # due to storing the binnmu changelog entry in deb822 buildinfo, the first
+ # character is an unwanted newline
+ $logentry =~ s/^\n//;
+ # while the linebreak at the beginning is wrong, there are two missing at
+ # the end
+ $logentry .= "\n\n";
+ push @binnmucmds,
+ '{ printf "%s" '
+ . (shellescape $logentry)
+ . "; cat debian/changelog; } > debian/changelog.debrebuild",
+ "mv debian/changelog.debrebuild debian/changelog";
+ }
+
+ my $build = '';
+ if ($build_archany and $build_archall) {
+ $build = "binary";
+ } elsif ($build_archany and !$build_archall) {
+ $build = "any";
+ } elsif (!$build_archany and $build_archall) {
+ $build = "all";
+ } else {
+ die "nothing to build\n";
+ }
+
+ my @cmd = (
+ 'env', '-i',
+ 'PATH=/usr/sbin:/usr/bin:/sbin:/bin',
+ 'mmdebstrap',
+ "--arch=$build_arch",
+ "--variant=apt",
+ '--aptopt=Acquire::Check-Valid-Until "false"',
+ '--aptopt=Acquire::http::Dl-Limit "1000";',
+ '--aptopt=Acquire::https::Dl-Limit "1000";',
+ '--aptopt=Acquire::Retries "5";',
+ '--include=' . (join ' ', @install),
+ '--essential-hook=chroot "$1" sh -c "'
+ . (
+ join ' && ',
+ 'rm /etc/apt/sources.list',
+ 'echo '
+ . (shellescape((join "\n", get_sources_list) . "\n"))
+ . ' >> /etc/apt/sources.list',
+ 'apt-get update'
+ )
+ . '"',
+ '--customize-hook=chroot "$1" sh -c "'
+ . (
+ join ' && ',
+ "apt-get source --only-source -d $srcpkgname=$srcpkgver",
+ "mkdir -p " . (shellescape(dirname $custom_build_path)),
+ "dpkg-source --no-check -x /"
+ . $srcpkg->get_basename(1) . '.dsc '
+ . (shellescape $custom_build_path),
+ 'cd ' . (shellescape $custom_build_path),
+ @binnmucmds,
+"env $environment dpkg-buildpackage -uc -a $host_arch --build=$build",
+ 'cd /',
+ 'rm -r ' . (shellescape $custom_build_path))
+ . '"',
+ '--customize-hook=sync-out '
+ . (dirname $custom_build_path)
+ . " $outdir",
+ $base_dist,
+ '/dev/null',
+ "deb $base_mirror/$build_date/ $base_dist main"
+ );
+ print(join ' ', @cmd) . "\n";
+
+ 0 == system @cmd or die "mmdebstrap failed\n";
} else {
- print " --no-arch-all";
+ die "unsupported builder: $builder\n";
}
-print " -d $base_dist";
-print " --no-run-lintian";
-if ($custom_build_path) {
- print " --build-path='${custom_build_path}'";
+
+# test if all checksums in the buildinfo file check out
+if ($builder ne "none") {
+ print "build artifacts stored in $outdir\n";
+
+ my $checksums = Dpkg::Checksums->new();
+ $checksums->add_from_control($cdata);
+ # remove the .dsc as we only did the binaries
+ # - the .dsc cannot be reproduced anyways because we cannot reproduce its
+ # signature
+ # - binNMUs can only be done with --build=any
+ foreach my $file ($checksums->get_files()) {
+ if ($file !~ /\.dsc$/) {
+ next;
+ }
+ $checksums->remove_file($file);
+ }
+
+ my $new_cdata
+ = Dpkg::Control->new(type => CTRL_FILE_BUILDINFO, allow_pgp => 1);
+ $new_cdata->load($new_buildinfo);
+ my $new_checksums = Dpkg::Checksums->new();
+ $new_checksums->add_from_control($new_cdata);
+
+ my @files = $checksums->get_files();
+ my @new_files = $new_checksums->get_files();
+
+ if (scalar @files != scalar @new_files) {
+ print("old buildinfo:\n" . (join "\n", @files) . "\n");
+ print("new buildinfo:\n" . (join "\n", @new_files) . "\n");
+ die "new buildinfo contains a different number of files\n";
+ }
+
+ for (my $i = 0 ; $i <= $#files ; $i++) {
+ if ($files[$i] ne $new_files[$i]) {
+ die "different checksum files at position $i\n";
+ }
+ if ($files[$i] =~ /\.dsc$/) {
+ print("skipping $files[$i]\n");
+ next;
+ }
+ print("checking $files[$i]: ");
+ if ($checksums->get_size($files[$i])
+ != $new_checksums->get_size($files[$i])) {
+ die "size differs for $files[$i]\n";
+ } else {
+ print("size... ");
+ }
+ my $chksum = $checksums->get_checksum($files[$i], undef);
+ my $new_chksum = $new_checksums->get_checksum($new_files[$i], undef);
+ if (scalar keys %{$chksum} != scalar keys %{$new_chksum}) {
+ die "different algos for $files[$i]\n";
+ }
+ foreach my $algo (keys %{$chksum}) {
+ if (!exists $new_chksum->{$algo}) {
+ die "$algo is not used in both buildinfo files\n";
+ }
+ if ($chksum->{$algo} ne $new_chksum->{$algo}) {
+ die "value of $algo differs for $files[$i]\n";
+ }
+ print("$algo... ");
+ }
+ print("all OK\n");
+ }
}
-print " $dsc_fname\n";
-print "BASE_DIST=$base_dist\n";
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/8e00fda376c6e64de5bd1ff3588a830d81e87c81
--
View it on GitLab: https://salsa.debian.org/qa/jenkins.debian.net/-/commit/8e00fda376c6e64de5bd1ff3588a830d81e87c81
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/qa-jenkins-scm/attachments/20201218/394affc5/attachment-0001.html>
More information about the Qa-jenkins-scm
mailing list