[Piuparts-devel] [Git][debian/piuparts][master] 13 commits: Start 1.5.2 development. Use `gbp dch --since=1.5.1 --multimaint-merge` to...
Nicolas Dandrimont (@olasd)
gitlab at salsa.debian.org
Thu Mar 20 21:28:12 GMT 2025
Nicolas Dandrimont pushed to branch master at Debian / piuparts
Commits:
f50b3c75 by Nicolas Dandrimont at 2024-11-15T16:24:49+01:00
Start 1.5.2 development. Use `gbp dch --since=1.5.1 --multimaint-merge` to generate a changelog for this release.
- - - - -
7bfd92ba by Nicolas Dandrimont at 2025-03-03T21:52:00+01:00
Ignore /var/cache/debconf/tmp.ci/
Closes: #1092667
- - - - -
bffee51f by Nicolas Dandrimont at 2025-03-03T21:54:53+01:00
Extend allow-database policy-rc.d for firebird4.0
Closes: #1095408
- - - - -
12781170 by Nicolas Dandrimont at 2025-03-03T22:20:46+01:00
Support versioned provides in metapackage generation
This is especially important on packages that conflict with a virtual
package they're providing, such as sysvinit-utils=3.14-3
- - - - -
70f4ed2f by Nicolas Dandrimont at 2025-03-03T22:25:52+01:00
Clean up some string concatenations
Gbp-dch: ignore
- - - - -
60e839cf by Nicolas Dandrimont at 2025-03-03T23:31:27+01:00
Deduplicate checks whether a file is ignored
- - - - -
6c349734 by Nicolas Dandrimont at 2025-03-03T23:32:05+01:00
Record and ignore files affected by systemd-tmpfiles
Closes: #1036399
- - - - -
1db83cba by Nicolas Dandrimont at 2025-03-03T23:55:55+01:00
Always use --allow-remove-essential on removals
apt-get will only remove packages that have been explicitly listed for
removal, which takes care of Protected: yes and Important: yes packages.
piuparts won't actually attempt to remove Essential packages as they're
part of the base chroot (hopefully).
Closes: #1010967, #1084173
- - - - -
d3b44248 by Nicolas Dandrimont at 2025-03-09T13:35:37+01:00
Remember systemd-tmpfiles when installing logrotate
Looks like under certain circumstances, installing logrotate in a
minimal chroot installs systemd, so the logrotate file check needs to
remember systemd tmpfiles if it hadn't been done earlier.
- - - - -
345d5865 by Nicolas Dandrimont at 2025-03-09T13:40:56+01:00
Make allow-remove-essential an explicit option for remove_packages
This can turn off essential package removal when we only try to remove
the package under test, in which case we usually ignore errors.
Thanks to Guillem Jover <guillem at debian.org> for the report.
- - - - -
7eb425d1 by Nicolas Dandrimont at 2025-03-17T11:33:28+01:00
Fix flake8 errors
Gbp-dch: ignore
- - - - -
5ef970c9 by Nicolas Dandrimont at 2025-03-20T22:05:05+01:00
Prepare changelog for piuparts 1.6.0
- - - - -
2c2d3d9a by Nicolas Dandrimont at 2025-03-20T21:27:59+00:00
Merge branch 'mr/release-1.6.0' into 'master'
Prepare changelog for piuparts 1.6.0
See merge request debian/piuparts!73
- - - - -
4 changed files:
- custom-scripts/scripts/post_remove_exceptions
- custom-scripts/scripts/pre_remove_exceptions
- debian/changelog
- piuparts.py
Changes:
=====================================
custom-scripts/scripts/post_remove_exceptions
=====================================
@@ -5,9 +5,6 @@ log_debug() {
echo "Debug: piuparts exception for package $PIUPARTS_OBJECTS"
}
-# cleanup from pre_remove_exceptions
-rm -fv /etc/apt/apt.conf.d/piuparts-allow-remove-essential
-
case ${PIUPARTS_OBJECTS%%=*} in
asclassic)
case "$PIUPARTS_DISTRIBUTION" in
=====================================
custom-scripts/scripts/pre_remove_exceptions
=====================================
@@ -60,32 +60,6 @@ case "$PIUPARTS_DISTRIBUTION" in
;;
esac
-case "$PIUPARTS_DISTRIBUTION" in
- lenny*|squeeze*|wheezy*|jessie*)
- # --allow-remove-essential was added in apt 1.1 (stretch)
- ;;
- *)
- case ${PIUPARTS_OBJECTS%%=*} in
- dracut*|\
- education-thin-client|\
- grub-efi-amd64-signed|\
- grub-efi-ia32-signed|\
- init|\
- ltsp-client-core-dbgsym|\
- ltsp-client-core|\
- ltsp-client)
- log_debug
- # requires removal of essential packages
- if [ ! -f /etc/apt/apt.conf.d/piuparts-allow-remove-essential ]
- then
- echo "Enabling APT::Get::allow-remove-essential"
- echo 'APT::Get::allow-remove-essential "true";' >> /etc/apt/apt.conf.d/piuparts-allow-remove-essential
- fi
- ;;
- esac
- ;;
-esac
-
# Allow removal of the kernel running on the host from the chroot.
UNAME_R="$(uname -r)"
echo "linux-image-$UNAME_R linux-image-$UNAME_R/prerm/removing-running-kernel-$UNAME_R boolean false" | debconf-set-selections
=====================================
debian/changelog
=====================================
@@ -1,3 +1,13 @@
+piuparts (1.6.0) unstable; urgency=medium
+
+ * Ignore /var/cache/debconf/tmp.ci/ (Closes: #1092667)
+ * Extend allow-database policy-rc.d for firebird4.0 (Closes: #1095408)
+ * Support versioned provides in metapackage generation
+ * Record and ignore files affected by systemd-tmpfiles (Closes: #1036399)
+ * Always use --allow-remove-essential on removals (Closes: #1010967, #1084173)
+
+ -- Nicolas Dandrimont <olasd at debian.org> Thu, 20 Mar 2025 22:02:58 +0100
+
piuparts (1.5.1) unstable; urgency=medium
[ Nicolas Dandrimont ]
=====================================
piuparts.py
=====================================
@@ -48,10 +48,10 @@ import tempfile
import time
import traceback
import uuid
-from collections import namedtuple
+from collections import defaultdict, namedtuple
from contextlib import ExitStack
from signal import SIGALRM, SIGKILL, SIGTERM, alarm, signal
-from typing import Dict
+from typing import Dict, Tuple
import apt_pkg
import distro_info
@@ -277,6 +277,7 @@ class Settings:
"/var/cache/debconf/templates.dat",
"/var/cache/debconf/templates.dat.old",
"/var/cache/debconf/templates.dat-old",
+ "/var/cache/debconf/tmp.ci/",
"/var/lib/apt/daily_lock",
"/var/lib/apt/extended_states",
"/var/lib/cdebconf/",
@@ -776,6 +777,7 @@ class Chroot:
self.mounts = []
self.initial_selections = None
self.avail_md5_history = []
+ self.systemd_tmpfiles: Dict[str, Tuple[str, str]] = {}
def create_temp_dir(self):
"""Create a temporary directory for the chroot."""
@@ -1148,6 +1150,7 @@ class Chroot:
policy += 'test "$1" = "postgresql-8.3" && exit 0\n'
policy += 'test "$1" = "firebird2.5-super" && exit 0\n'
policy += 'test "$1" = "firebird3.0" && exit 0\n'
+ policy += 'test "$1" = "firebird4.0" && exit 0\n'
policy += "exit 101\n"
create_file(full_name, policy)
os.chmod(full_name, 0o755)
@@ -1290,6 +1293,40 @@ class Chroot:
"""Remember initial selections to easily recognize mismatching chroot metadata"""
self.initial_selections = self.get_selections()
+ def remember_systemd_tmpfiles(self):
+ """Remember files potentially affected by systemd-tmpfiles.
+
+ Only files and directories with some flags are ignored by piuparts."""
+ ignored_flags = (
+ "f", # Create file
+ "w", # Write file contents
+ "d", # Create directory
+ "D", # Create directory (and remove on --remove)
+ "p", # Create named pipe
+ "L", # Create symlink
+ "c", # Create character device
+ "b", # Create block device
+ "C", # Copy file or directory
+ )
+ (retcode, output) = self.run(["systemd-tmpfiles", "--cat-config"], ignore_errors=True)
+ if retcode != 0:
+ return
+
+ current_file = "unknown tmpfiles.d snippet"
+
+ for line in output.splitlines():
+ line = line.strip()
+ if not line:
+ continue
+ if line.startswith("#"):
+ if line.startswith("# /"):
+ if os.path.exists(self.relative(line[3:])):
+ current_file = line[2:]
+ continue
+ flag, filename, *_ = line.split()
+ if flag[0] in ignored_flags:
+ self.systemd_tmpfiles[filename] = (current_file, flag)
+
def upgrade_to_distros(self, distros, packages, apt_get_upgrade=False):
"""Upgrade a chroot installation to each successive distro."""
for distro in distros:
@@ -1353,7 +1390,7 @@ class Chroot:
def list_installed_files(self, pre_info, post_info):
"""List the new files installed, removed and modified between two dir trees.
Actually, it is a nice output of the funcion diff_meta_dat."""
- (new, removed, modified) = diff_meta_data(pre_info, post_info)
+ (new, removed, modified) = diff_meta_data(pre_info, post_info, self.is_ignored)
file_owners = self.get_files_owned_by_packages()
if new:
@@ -1639,11 +1676,14 @@ class Chroot:
else:
logging.info(msg)
- def remove_packages(self, packages, ignore_errors=False):
+ def remove_packages(self, packages, allow_remove_essential=True, ignore_errors=False):
"""Remove packages in a chroot."""
+ base_command = ["apt-get", "remove"]
+ if allow_remove_essential:
+ base_command.append("--allow-remove-essential")
if packages:
self.run(
- ["apt-get", "remove"] + ["%s-" % x if x.endswith("+") else x for x in unqualify(packages)],
+ base_command + ["%s-" % x if x.endswith("+") else x for x in unqualify(packages)],
ignore_errors=ignore_errors,
)
@@ -1929,20 +1969,27 @@ class Chroot:
if selinux_enabled():
self.mount("/sys/fs/selinux", self.selinuxfs_path(), opts=["bind", "ro"])
- def is_ignored(self, pathname, info="PATH"):
+ def is_ignored(self, pathname, info="PATH", quiet=False):
"""Is a file (or dir or whatever) to be ignored?"""
if pathname in settings.ignored_files:
return True
if ":" + pathname in settings.ignored_files:
- logging.info("IGNORED %s: %s" % (info, pathname))
+ if not quiet:
+ logging.info("IGNORED %s: %s" % (info, pathname))
+ return True
+ stripped_pathname = pathname.rstrip("/")
+ if stripped_pathname in self.systemd_tmpfiles:
+ snippet, flag = self.systemd_tmpfiles[stripped_pathname]
+ if not quiet:
+ logging.info("IGNORED %s: %s (flag %s systemd-tmpfiles snippet %s)", info, pathname, flag, snippet)
return True
for pattern in settings.ignored_patterns:
if pattern[0] == ":":
- verbose = True
+ verbose = not quiet
pattern = pattern[1:]
else:
verbose = False
- if re.search("^" + pattern + "$", pathname):
+ if re.fullmatch(pattern, pathname):
if verbose:
logging.info("IGNORED %s: %s" % (info, pathname))
return True
@@ -2095,6 +2142,9 @@ class Chroot:
list of packages that were installed"""
old_selections = self.get_selections()
self.run(["apt-get", "install", "-y", "logrotate"])
+ # If systemd wasn't installed yet, logrotate installs it, so we get to
+ # update our list of known systemd-tmpfiles cruft
+ self.remember_systemd_tmpfiles()
diff = diff_selections(self, old_selections)
return diff.keys()
@@ -2178,50 +2228,22 @@ def format_object_attributes(obj):
return res
-def diff_meta_data(tree1, tree2, quiet=False):
+def diff_meta_data(tree1, tree2, is_ignored, quiet=False):
"""Compare two dir trees and return list of new files (only in 'tree2'),
removed files (only in 'tree1'), and modified files."""
tree1_c = tree1.copy()
tree2_c = tree2.copy()
- for name in settings.ignored_files:
- if name[0] == ":":
- verbose = not quiet
- name = name[1:]
- else:
- verbose = False
- if name in tree1_c:
- if verbose:
- logging.info("IGNORED PATH at 1: %s" % name)
+ for name in tree1:
+ if is_ignored(name, info="PATH at 1", quiet=quiet):
del tree1_c[name]
- if name in tree2_c:
- if verbose:
- logging.info("IGNORED PATH at 2: %s" % name)
+ for name in tree2:
+ if is_ignored(name, info="PATH at 2", quiet=quiet):
del tree2_c[name]
- for pattern in settings.ignored_patterns:
- if pattern[0] == ":":
- verbose = not quiet
- pattern = pattern[1:]
- else:
- verbose = False
- pat = re.compile(pattern)
- for name in tree1.keys():
- m = pat.search(name)
- if m and name in tree1_c:
- if verbose:
- logging.info("IGNORED PATH at 1: %s" % name)
- del tree1_c[name]
- for name in tree2.keys():
- m = pat.search(name)
- if m and name in tree2_c:
- if verbose:
- logging.info("IGNORED PATH at 2: %s" % name)
- del tree2_c[name]
-
modified = []
- for name in tree1.keys():
+ for name in tree1:
if name in tree1_c and name in tree2_c:
if objects_are_different(tree1[name], tree2[name]):
if not quiet:
@@ -2410,15 +2432,15 @@ def check_results(chroot, chroot_state, file_owners, deps_info=None):
current_info = chroot.get_tree_meta_data()
if settings.warn_on_others and deps_info is not None:
- (new, removed, modified) = diff_meta_data(reference_info, current_info)
- (depsnew, depsremoved, depsmodified) = diff_meta_data(reference_info, deps_info, quiet=True)
+ (new, removed, modified) = diff_meta_data(reference_info, current_info, chroot.is_ignored)
+ (depsnew, depsremoved, depsmodified) = diff_meta_data(reference_info, deps_info, chroot.is_ignored, quiet=True)
warnnew = prune_files_list(new, depsnew)
warnremoved = prune_files_list(removed, depsremoved)
warnmodified = prune_files_list(modified, depsmodified)
else:
- (new, removed, modified) = diff_meta_data(reference_info, current_info)
+ (new, removed, modified) = diff_meta_data(reference_info, current_info, chroot.is_ignored)
if new:
if settings.warn_on_leftovers_after_purge:
@@ -2525,9 +2547,14 @@ def install_purge_test(chroot, chroot_state, package_files, packages, extra_pack
arch = a
if arch != a:
logging.info("architecture mismatch: %s != %s)" % (arch, a))
+ unversioned_conflicts = defaultdict(list)
+ for conflict in conflicts:
+ unversioned_conflicts[conflict.split("(")[0].strip()].append(conflict)
for provided in provides:
- if provided in conflicts:
- conflicts.remove(provided)
+ unversioned_provide = provided.split("(")[0].strip()
+ if unversioned_provide in unversioned_conflicts:
+ for c in unversioned_conflicts[unversioned_provide]:
+ conflicts.remove(c)
all_depends = ", ".join(depends)
all_conflicts = ", ".join(conflicts)
metapackage = make_metapackage(
@@ -2572,6 +2599,8 @@ def install_purge_test(chroot, chroot_state, package_files, packages, extra_pack
chroot.run_scripts("post_install")
+ chroot.remember_systemd_tmpfiles()
+
if settings.install_purge_install:
file_owners = chroot.get_files_owned_by_packages()
chroot.restore_selections(chroot_state_with_deps, packages)
@@ -2585,7 +2614,7 @@ def install_purge_test(chroot, chroot_state, package_files, packages, extra_pack
chroot.install_packages(package_files, packages, with_scripts=True)
if settings.install_remove_install:
- chroot.remove_packages(packages, ignore_errors=True)
+ chroot.remove_packages(packages, allow_remove_essential=False, ignore_errors=True)
logging.info("Reinstalling after remove")
chroot.install_packages(package_files, packages, with_scripts=True)
chroot.install_packages(package_files, packages, with_scripts=True, reinstall=True)
@@ -2627,7 +2656,7 @@ def install_upgrade_test(chroot, chroot_state, package_files, packages, old_pack
file_owners_before = chroot.get_files_owned_by_packages()
if settings.install_remove_install:
- chroot.remove_packages(packages, ignore_errors=True)
+ chroot.remove_packages(packages, allow_remove_essential=False, ignore_errors=True)
# Then from the package files.
os.environ["PIUPARTS_PHASE"] = "upgrade"
@@ -2779,7 +2808,7 @@ def install_and_upgrade_between_distros(package_files, packages_qualified):
chroot.install_packages_by_name(known_packages)
if settings.install_remove_install:
- chroot.remove_packages(packages, ignore_errors=True)
+ chroot.remove_packages(packages, allow_remove_essential=False, ignore_errors=True)
distupgrade_packages = []
chroot.check_for_no_processes()
@@ -2878,7 +2907,7 @@ def parse_command_line():
"--apt",
action="store_true",
default=False,
- help="Command line arguments are package names " + "to be installed via apt.",
+ help="Command line arguments are package names to be installed via apt.",
)
parser.add_option(
@@ -2998,7 +3027,7 @@ def parse_command_line():
"-e",
"--existing-chroot",
metavar="DIR",
- help="Use DIR as the contents of the initial " + "chroot, instead of building a new one with " + "debootstrap",
+ help="Use DIR as the contents of the initial chroot, instead of building a new one with debootstrap",
)
parser.add_option(
@@ -3015,7 +3044,7 @@ def parse_command_line():
action="append",
metavar="FILENAME",
default=[],
- help="Add FILENAME to list of filenames to be " + "ignored when comparing changes to chroot."
+ help="Add FILENAME to list of filenames to be ignored when comparing changes to chroot."
"FILENAMES prefixed with ':' will be reported verbosely.",
)
@@ -3068,7 +3097,7 @@ def parse_command_line():
"--keep-sources-list",
action="store_true",
default=False,
- help="Don't modify the chroot's " + "etc/apt/sources.list.",
+ help="Don't modify the chroot's etc/apt/sources.list.",
)
parser.add_option(
@@ -3076,14 +3105,14 @@ def parse_command_line():
"--log-file",
"--logfile",
metavar="FILENAME",
- help="Write log file to FILENAME in addition to " + "the standard output.",
+ help="Write log file to FILENAME in addition to the standard output.",
)
parser.add_option(
"--list-installed-files",
action="store_true",
default=False,
- help="List files added to the chroot after the " + "installation of the package.",
+ help="List files added to the chroot after the installation of the package.",
)
parser.add_option(
@@ -3100,7 +3129,7 @@ def parse_command_line():
metavar="SNAPSHOT-SIZE",
action="store",
default="1G",
- help="Use SNAPSHOT-SIZE as snapshot size when creating " + "a new LVM snapshot (default: 1G)",
+ help="Use SNAPSHOT-SIZE as snapshot size when creating a new LVM snapshot (default: 1G)",
)
parser.add_option(
@@ -3165,7 +3194,7 @@ def parse_command_line():
"--no-ignores",
action="callback",
callback=forget_ignores,
- help="Forget all ignores set so far, including " + "built-in ones.",
+ help="Forget all ignores set so far, including built-in ones.",
)
parser.add_option(
@@ -3183,7 +3212,7 @@ def parse_command_line():
"--no-upgrade-test",
action="store_true",
default=False,
- help="Skip testing the upgrade from an existing version " + "in the archive.",
+ help="Skip testing the upgrade from an existing version in the archive.",
)
parser.add_option(
@@ -3232,7 +3261,7 @@ def parse_command_line():
"--pbuilder",
action="callback",
callback=set_basetgz_to_pbuilder,
- help="Use /var/cache/pbuilder/base.tgz as the base " + "tarball.",
+ help="Use /var/cache/pbuilder/base.tgz as the base tarball.",
)
parser.add_option(
@@ -3323,7 +3352,7 @@ def parse_command_line():
)
parser.add_option(
- "-t", "--tmpdir", metavar="DIR", help="Use DIR for temporary storage. Default is " + "$TMPDIR or /tmp."
+ "-t", "--tmpdir", metavar="DIR", help="Use DIR for temporary storage. Default is $TMPDIR or /tmp."
)
parser.add_option(
@@ -3555,7 +3584,7 @@ def parse_command_line():
exitcode = 1
if not args:
- logging.error("Need command line arguments: " + "names of packages or package files")
+ logging.error("Need command line arguments: names of packages or package files")
exitcode = 1
settings.testobjects = args
View it on GitLab: https://salsa.debian.org/debian/piuparts/-/compare/82912582ddb724d11766e945b23fc0159853befe...2c2d3d9a850402f254ae120bd3811ba58cab3f3a
--
View it on GitLab: https://salsa.debian.org/debian/piuparts/-/compare/82912582ddb724d11766e945b23fc0159853befe...2c2d3d9a850402f254ae120bd3811ba58cab3f3a
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/piuparts-devel/attachments/20250320/92ae4195/attachment-0001.htm>
More information about the Piuparts-devel
mailing list