[Piuparts-devel] [Git][debian/piuparts][master] 20 commits: Remove oldold*stable dists from json summary
Nicolas Dandrimont (@olasd)
gitlab at salsa.debian.org
Wed Jan 11 12:32:33 GMT 2023
Nicolas Dandrimont pushed to branch master at Debian / piuparts
Commits:
adfef741 by David Steele at 2021-10-23T12:50:11-04:00
Remove oldold*stable dists from json summary
Among other things, distro_info reports them as "unknown".
- - - - -
24e89b75 by Holger Levsen at 2022-02-02T10:12:37+01:00
custom-scripts/scripts/pre_test_root_password: fix grammar in comment
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
190399be by Holger Levsen at 2022-02-02T10:13:31+01:00
custom-scripts/scripts/pre_test_root_password: further improve grammar in comment
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
e7538053 by Luca Boccassi at 2022-07-13T10:36:42+01:00
piuparts: detect files moving between / and /usr on upgrade
The TC issued a recommendation to avoid moving files between
bin|sbin|lib* and /usr/bin|sbin|lib* for the Bookworm cycle.
Detect such moves and raise an error.
For more details see:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994388#80
- - - - -
4489752a by Luca Boccassi at 2022-07-13T10:36:42+01:00
piuparts: detect files moving between / and /usr on dist-upgrade
Same as the check on package install/upgrade, but across distribution
dist-upgrades.
- - - - -
59c00b8b by Luca Boccassi at 2022-07-13T10:36:42+01:00
p.conf: enable --warn-on-usr-move for bullseye/bookworm
- - - - -
92c1faff by Nicolas Dandrimont at 2022-07-13T20:06:49+02:00
Merge branch 'usrmerge' into develop
piuparts: detect files moving between / and /usr on upgrade
See merge request https://salsa.debian.org/debian/piuparts/-/merge_requests/37
- - - - -
98b4fdcd by Luca Boccassi at 2022-07-17T20:17:08+01:00
custom_scripts: ensure usr-is-merged/usrmerge does not fail the migration test
The new usr-is-merged metapackage intentionally fails to install
on systems that are not merged-usr, unless a specific override
flag is passed.
usrmerge does a live-upgrade of the chroot, which piuparts does not
expect.
Use the skip flag, so that migration is not blocked.
Debootstrap will start using this flag once the package is
available in testing. For details see:
https://sources.debian.org/src/usrmerge/27/debian/README.Debian/#L66
and:
https://lists.debian.org/debian-ctte/2022/07/msg00019.html
- - - - -
c3352499 by Nicolas Dandrimont at 2022-07-17T22:06:03+02:00
Merge branch 'usr-is-merged' into 'develop'
custom_scripts: ensure usr-is-merged does not fail the migration test
See merge request https://salsa.debian.org/debian/piuparts/-/merge_requests/38
- - - - -
67a85a4f by David Steele at 2022-11-02T13:14:39-04:00
Add Stable Release tasks to README_pejacevic.txt
... to update summary.json for the new distribution environment.
- - - - -
3331b140 by Nicolas Dandrimont at 2023-01-11T11:35:00+01:00
piuparts-slave: run isort
- - - - -
ad7e30b3 by Nicolas Dandrimont at 2023-01-11T11:59:39+01:00
piuparts-slave: replace pipes.quote with shlex.quote
- - - - -
b9ca98d1 by Nicolas Dandrimont at 2023-01-11T11:59:52+01:00
piuparts.py: run isort
- - - - -
fc0047c9 by Nicolas Dandrimont at 2023-01-11T11:59:52+01:00
piuparts.py: replace pipes.quote with shlex.quote
- - - - -
2cfae2aa by Nicolas Dandrimont at 2023-01-11T12:00:00+01:00
test_piuparts: replace mox3 with unittest.mock
Closes: #1028453
- - - - -
2fd10804 by Nicolas Dandrimont at 2023-01-11T12:05:53+01:00
Migrate lintian overrides to pointed hints
- - - - -
cd45cc8f by Nicolas Dandrimont at 2023-01-11T13:19:53+01:00
Simplify docs install in Makefile
- - - - -
224e9beb by Nicolas Dandrimont at 2023-01-11T13:25:39+01:00
Use sphinxdoc dh plugin
- - - - -
0ac90edf by Nicolas Dandrimont at 2023-01-11T13:25:39+01:00
Update changelog
- - - - -
fbca117e by Nicolas Dandrimont at 2023-01-11T13:31:48+01:00
Release to unstable
- - - - -
17 changed files:
- Makefile
- custom-scripts/scripts/post_purge_exceptions
- custom-scripts/scripts/pre_install_exceptions
- custom-scripts/scripts/pre_test_root_password
- debian/changelog
- debian/control
- debian/piuparts.lintian-overrides
- debian/rules
- docs/README_pejacevic.txt
- docs/piuparts/piuparts.1.txt
- instances/piuparts.conf-template.pejacevic
- + known_problems/file_moved_usr_error.conf
- + known_problems/file_moved_usr_issue.conf
- piuparts-report.py
- piuparts-slave.py
- piuparts.py
- tests/test_piuparts.py
Changes:
=====================================
Makefile
=====================================
@@ -83,42 +83,7 @@ install-doc: build-stamp
install -d $(DESTDIR)$(docdir)/
install -m 0644 docs/README.txt docs/README_server.txt $(DESTDIR)$(docdir)/
# html
- install -d $(DESTDIR)$(docdir)/html/
- install -m 0644 docs/build/*.html $(DESTDIR)$(docdir)/html/
- install -m 0644 docs/build/searchindex.js $(DESTDIR)$(docdir)/html/
- install -m 0644 docs/build/objects.inv $(DESTDIR)$(docdir)/html/
- install -d $(DESTDIR)$(docdir)/html/_static/
- install -m 0644 docs/build/_static/* $(DESTDIR)$(docdir)/html/_static
- install -d $(DESTDIR)$(docdir)/html/piuparts/
- install -m 0644 docs/build/piuparts/index.html $(DESTDIR)$(docdir)/html/piuparts/
- install -m 0644 docs/build/piuparts/piuparts.1.html $(DESTDIR)$(docdir)/html/piuparts/
- install -d $(DESTDIR)$(docdir)/html/piuparts_slave_run/
- install -m 0644 docs/build/piuparts_slave_run/index.html $(DESTDIR)$(docdir)/html/piuparts_slave_run/
- install -m 0644 docs/build/piuparts_slave_run/piuparts_slave_run.8.html $(DESTDIR)$(docdir)/html/piuparts_slave_run/
- install -d $(DESTDIR)$(docdir)/html/piuparts_slave_join/
- install -m 0644 docs/build/piuparts_slave_join/index.html $(DESTDIR)$(docdir)/html/piuparts_slave_join/
- install -m 0644 docs/build/piuparts_slave_join/piuparts_slave_join.8.html $(DESTDIR)$(docdir)/html/piuparts_slave_join/
- install -d $(DESTDIR)$(docdir)/html/piuparts_slave_stop/
- install -m 0644 docs/build/piuparts_slave_stop/index.html $(DESTDIR)$(docdir)/html/piuparts_slave_stop/
- install -m 0644 docs/build/piuparts_slave_stop/piuparts_slave_stop.8.html $(DESTDIR)$(docdir)/html/piuparts_slave_stop/
- install -d $(DESTDIR)$(docdir)/html/
- install -m 0644 docs/build/*.html $(DESTDIR)$(docdir)/html/
- install -m 0644 docs/build/searchindex.js $(DESTDIR)$(docdir)/html/
- install -m 0644 docs/build/objects.inv $(DESTDIR)$(docdir)/html/
- install -d $(DESTDIR)$(docdir)/html/_static/
- install -m 0644 docs/build/_static/* $(DESTDIR)$(docdir)/html/_static
- install -d $(DESTDIR)$(docdir)/html/piuparts/
- install -m 0644 docs/build/piuparts/index.html $(DESTDIR)$(docdir)/html/piuparts/
- install -m 0644 docs/build/piuparts/piuparts.1.html $(DESTDIR)$(docdir)/html/piuparts/
- install -d $(DESTDIR)$(docdir)/html/piuparts_slave_run/
- install -m 0644 docs/build/piuparts_slave_run/index.html $(DESTDIR)$(docdir)/html/piuparts_slave_run/
- install -m 0644 docs/build/piuparts_slave_run/piuparts_slave_run.8.html $(DESTDIR)$(docdir)/html/piuparts_slave_run/
- install -d $(DESTDIR)$(docdir)/html/piuparts_slave_join/
- install -m 0644 docs/build/piuparts_slave_join/index.html $(DESTDIR)$(docdir)/html/piuparts_slave_join/
- install -m 0644 docs/build/piuparts_slave_join/piuparts_slave_join.8.html $(DESTDIR)$(docdir)/html/piuparts_slave_join/
- install -d $(DESTDIR)$(docdir)/html/piuparts_slave_stop/
- install -m 0644 docs/build/piuparts_slave_stop/index.html $(DESTDIR)$(docdir)/html/piuparts_slave_stop/
- install -m 0644 docs/build/piuparts_slave_stop/piuparts_slave_stop.8.html $(DESTDIR)$(docdir)/html/piuparts_slave_stop/
+ cp -r docs/build/ $(DESTDIR)$(docdir)/html/
# manpages
install -d $(DESTDIR)$(man1dir)
install -m 0644 piuparts.1 $(DESTDIR)$(man1dir)/
=====================================
custom-scripts/scripts/post_purge_exceptions
=====================================
@@ -53,6 +53,13 @@ case ${PIUPARTS_OBJECTS%%=*} in
log_debug
sleep 10
;;
+ *usr-is-merged*|*usrmerge*)
+ # Remove the exception flag that we added in pre_install_exceptions, if we added it
+ if [ -f /etc/unsupported-skip-usrmerge-conversion.piuparts ]; then
+ log_debug
+ rm -f /etc/unsupported-skip-usrmerge-conversion*
+ fi
+ ;;
esac
if [ -f /var/run/piuparts-stamp-dkms-installed ]; then
=====================================
custom-scripts/scripts/pre_install_exceptions
=====================================
@@ -106,6 +106,20 @@ case ${PIUPARTS_OBJECTS%%=*} in
mkdir -p /var/state/samhain/
touch /var/state/samhain/samhain_file
;;
+ *usr-is-merged*|*usrmerge*)
+ # usr-is-merged intentionally fails to install unless either the system is merged-usr, or
+ # the exception flag is set.
+ # usrmerge does a live-upgrade of the chroot, which piuparts does not expect, so skip it.
+ # debootstrap will take care of this in a future update, so remember if we were the ones
+ # that added the flag file, to remove it in the post_purge_exceptions script.
+ # For details on the flag file, see: https://sources.debian.org/src/usrmerge/27/debian/README.Debian/#L66
+ # and: https://lists.debian.org/debian-ctte/2022/07/msg00019.html
+ if [ ! -f /etc/unsupported-skip-usrmerge-conversion ]; then
+ log_debug
+ touch /etc/unsupported-skip-usrmerge-conversion.piuparts
+ echo "this system will not be supported in the future" > /etc/unsupported-skip-usrmerge-conversion
+ fi
+ ;;
esac
if [ "$PIUPARTS_DISTRIBUTION" = "jessie" ]; then
=====================================
custom-scripts/scripts/pre_test_root_password
=====================================
@@ -2,6 +2,6 @@
set -e
# sudo refuses removal if no root password is set, so set one
-# do this unconditionally, as there are quite some packages depending on sudo
+# unconditionally, as there are quite some packages depending on sudo
# (and since its harmless and fast)
yes "yes" 2>/dev/null | passwd
=====================================
debian/changelog
=====================================
@@ -1,3 +1,33 @@
+piuparts (1.1.6) unstable; urgency=medium
+
+ [ David Steele ]
+ * Remove oldold*stable dists from json summary
+
+ [ Holger Levsen ]
+ * custom-scripts/scripts/pre_test_root_password: fix grammar in comment
+
+ [ Luca Boccassi ]
+ * piuparts: detect files moving between / and /usr on upgrade
+ * piuparts: detect files moving between / and /usr on dist-upgrade
+ * p.conf: enable --warn-on-usr-move for bullseye/bookworm
+ * custom_scripts: ensure usr-is-merged/usrmerge does not fail the
+ migration test
+
+ [ David Steele ]
+ * Add Stable Release tasks to README_pejacevic.txt
+
+ [ Nicolas Dandrimont ]
+ * piuparts-slave: run isort
+ * piuparts-slave: replace pipes.quote with shlex.quote
+ * piuparts.py: run isort
+ * piuparts.py: replace pipes.quote with shlex.quote
+ * test_piuparts: replace mox3 with unittest.mock (Closes: #1028453)
+ * Migrate lintian overrides to pointed hints
+ * Simplify docs install in Makefile
+ * Use sphinxdoc dh plugin
+
+ -- Nicolas Dandrimont <olasd at debian.org> Wed, 11 Jan 2023 13:31:41 +0100
+
piuparts (1.1.5) unstable; urgency=medium
[ Nicolas Dandrimont ]
@@ -10,8 +40,8 @@ piuparts (1.1.5) unstable; urgency=medium
* Remove oldstable results from summary.json.
[ Athos Ribeiro ]
- * d/rules: set GO111MODULE to auto to maintain pre Go 1.16 behavior. Closes: #991970.
- (MR: debian/piuparts!35)
+ * d/rules: set GO111MODULE to auto to maintain pre Go 1.16 behavior.
+ Closes: #991970. (MR: debian/piuparts!35)
-- Holger Levsen <holger at debian.org> Thu, 14 Oct 2021 15:23:26 +0200
=====================================
debian/control
=====================================
@@ -15,7 +15,6 @@ Build-Depends:
python3-debian,
python3-debianbts (>= 2.10.0),
python3-distro-info,
- python3-mox3,
python3-nose,
python3-six,
python3-sphinx,
@@ -35,13 +34,13 @@ Depends:
# keep this list in sync with piuparts-slave-from-git-deps
debootstrap (>= 1.0.89~),
debsums (>= 2.2.2~),
- libjs-sphinxdoc,
lsb-release,
lsof,
mount,
python3-debian,
${python3:Depends},
${misc:Depends},
+ ${sphinxdoc:Depends}
Recommends:
adequate
Suggests:
=====================================
debian/piuparts.lintian-overrides
=====================================
@@ -1,6 +1,6 @@
# it is impossible not to do this without losing compatibility with ancient releases
-uses-dpkg-database-directly etc/piuparts/scripts-debug-purge/post_remove_postrm_set-x
-uses-dpkg-database-directly etc/piuparts/scripts-debug-remove/pre_remove_prerm_postrm_set-x
-uses-dpkg-database-directly etc/piuparts/scripts/post_remove_exceptions
-uses-dpkg-database-directly etc/piuparts/scripts/pre_remove_40_find_missing_md5sums
-uses-dpkg-database-directly usr/sbin/piuparts
+uses-dpkg-database-directly [etc/piuparts/scripts-debug-purge/post_remove_postrm_set-x]
+uses-dpkg-database-directly [etc/piuparts/scripts-debug-remove/pre_remove_prerm_postrm_set-x]
+uses-dpkg-database-directly [etc/piuparts/scripts/post_remove_exceptions]
+uses-dpkg-database-directly [etc/piuparts/scripts/pre_remove_40_find_missing_md5sums]
+uses-dpkg-database-directly [usr/sbin/piuparts]
=====================================
debian/rules
=====================================
@@ -7,7 +7,7 @@ export GOCACHE := $(CURDIR)/.gocache
export GO111MODULE := auto
%:
- dh $@ --with python3
+ dh $@ --with python3,sphinxdoc
override_dh_auto_build:
$(MAKE) prefix=/usr build build-doc
=====================================
docs/README_pejacevic.txt
=====================================
@@ -226,11 +226,27 @@ daily by '~piupartsm/crontab'.
More checks should be added as we become aware of them.
+:ref:`top <top3>`
+
+:blue:`New Stable Release`
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The server exports data, used by DDPO et al. to display test results by
+package. With a new release of Debian Stable, a number of tasks are required to
+update the summary data appropriately:
+
+- Edit 'piuparts.conf.pejacevic' to set 'json-sections = none' for the sections
+ that now represent oldstable ('foo', 'bar2foo', 'foo-proposed', etc.).
+- Update the 'distro-info-data' package, so that 'piuparts-report' can correctly
+ assign sections to distribution names.
+
+
:ref:`top <top3>`
:blue:`Authors`
^^^^^^^^^^^^^^^
-Last updated: February 2017
+Last updated: November 2022
Holger Levsen <holger at layer-acht.org>
+David Steele <steele at debian.org>
=====================================
docs/piuparts/piuparts.1.txt
=====================================
@@ -330,6 +330,10 @@ Options must come before the other command line arguments.
Behavior with multiple packages given on the command-line could be problematic, particularly if the dependency tree of one package in the list includes another in the list. Therefore, it is recommended to use this option with one package at a time.
+*-*-warn-on-usr-move*='disabled|warn|fail'::
+ Whether to enable the test (with a warning or a failure) that checks if files are moved between /{bin|sbin|lib*} and /usr/{bin|sbin|lib*}.
+ Accepted values: 'disabled' (default), 'warn', 'fail'.
+
EXAMPLES
=====================================
instances/piuparts.conf-template.pejacevic
=====================================
@@ -40,15 +40,18 @@ flags-end-oldstable = %(flags-end-buster)s
# common flags for tests starting in bookworm
flags-start-bookworm =
-# no flags needed
+# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994388#80
+ --warn-on-usr-move fail
# common flags for tests ending in bookworm
flags-end-bookworm =
-# no flags needed
+# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994388#80
+ --warn-on-usr-move fail
# common flags for tests starting in bullseye
flags-start-bullseye =
-# no flags needed
+# https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994388#80
+ --warn-on-usr-move fail
# common flags for tests ending in bullseye
flags-end-bullseye =
@@ -891,6 +894,7 @@ upgrade-test-distros = jessie jessie-backports
[jessie]
precedence = 7
+json-sections = none
piuparts-flags =
%(flags-default)s
%(flags-start-jessie)s
=====================================
known_problems/file_moved_usr_error.conf
=====================================
@@ -0,0 +1,13 @@
+# detect a file moving between bin/sbin/lib* and usr/bin|sbin|lib*
+#
+PATTERN='(WARN|FAIL): File\(s\) moved between /.* and /usr/.*:'
+WHERE='fail bugged affected'
+ISSUE=0
+HEADER='File(s) moved between /{bin|sbin|lib*} and /usr/{bin|sbin|lib*}'
+HELPTEXT='
+<p>
+The Technical Committee recommended against moving files between between /{bin|sbin|lib*}
+and /usr/{bin|sbin|lib*} during the Bookworm development cycle. See
+<a href="https://bugs.debian.org/994388#80">Debian bug #994388.</a>
+</p>
+'
=====================================
known_problems/file_moved_usr_issue.conf
=====================================
@@ -0,0 +1,13 @@
+# detect a file moving between bin/sbin/lib* and usr/bin|sbin|lib*
+#
+PATTERN='(WARN|FAIL): File\(s\) moved between /.* and /usr/.*:'
+WHERE='pass'
+ISSUE=1
+HEADER='File(s) moved between /{bin|sbin|lib*} and /usr/{bin|sbin|lib*}'
+HELPTEXT='
+<p>
+The Technical Committee recommended against moving files between between /{bin|sbin|lib*}
+and /usr/{bin|sbin|lib*} during the Bookworm development cycle. See
+<a href="https://bugs.debian.org/994388#80">Debian bug #994388.</a>
+</p>
+'
=====================================
piuparts-report.py
=====================================
@@ -501,6 +501,8 @@ linktarget_by_template = [
("missing_md5sums_error.tpl", "...and logfile reports missing md5sums"),
("unowned_lib_symlink_error.tpl", "...and logfile reports unowned lib symlinks"),
("piuparts-depends-dummy_error.tpl", "...and logfile reports piuparts-depends-dummy.deb could not be installed"),
+ ("file_moved_usr_error,tpl", "...and logfile reports a file moved between /{bin|sbin|lib*} and /usr/{bin|sbin|lib*}"),
+ ("file_moved_usr_issue,tpl", "but logfile reports a file moved between /{bin|sbin|lib*} and /usr/{bin|sbin|lib*}"),
("unclassified_failures.tpl", "due to unclassified failures"),
]
=====================================
piuparts-slave.py
=====================================
@@ -24,17 +24,18 @@ Lars Wirzenius <liw at iki.fi>
"""
from __future__ import print_function
+import fcntl
+import logging
import os
-import sys
+import random
+import shlex
import stat
-import time
-import logging
-from signal import alarm, signal, SIGALRM, SIGINT, SIGKILL, SIGHUP, SIGUSR1
import subprocess
-import fcntl
-import random
+import sys
+import time
+from signal import SIGALRM, SIGHUP, SIGINT, SIGKILL, SIGUSR1, alarm, signal
+
import apt_pkg
-import pipes
import piupartslib.conf
import piupartslib.packagesdb
@@ -851,7 +852,7 @@ def log_name(package, version):
def command2string(command):
"""Quote s.t. copy+paste from the logfile gives a runnable command in the shell."""
- return " ".join([pipes.quote(arg) for arg in command])
+ return " ".join([shlex.quote(arg) for arg in command])
def run_test_with_timeout(cmd, maxwait, kill_all=True):
=====================================
piuparts.py
=====================================
@@ -35,38 +35,38 @@ from __future__ import print_function
VERSION = "__PIUPARTS_VERSION__"
-import distro_info
-import time
+import json
import logging
import optparse
-import sys
-import tempfile
-import shutil
import os
-import tarfile
-import stat
-import re
-import json
+import pathlib
import pickle
+import re
+import shlex
+import shutil
+import stat
import subprocess
+import sys
+import tarfile
+import tempfile
+import time
import traceback
import uuid
-import apt_pkg
-import pipes
from collections import namedtuple
-from signal import alarm, signal, SIGALRM, SIGTERM, SIGKILL
+from signal import SIGALRM, SIGKILL, SIGTERM, alarm, signal
+
+import apt_pkg
+import distro_info
try:
from debian import deb822
except ImportError:
from debian_bundle import deb822
-import piupartslib.conf
-
-from six.moves import urllib
-
import six
+from six.moves import urllib
+import piupartslib.conf
apt_pkg.init_system()
@@ -220,6 +220,7 @@ class Settings:
self.warn_on_debsums_errors = False
self.warn_on_install_over_symlink = False
self.warn_if_inadequate = True
+ self.warn_on_usr_move = "disabled"
self.pedantic_purge_test = False
self.ignored_files = [
# /root/.rnd should *not* be listed here, see #750099
@@ -504,7 +505,7 @@ def indent_string(str):
def command2string(command):
"""Quote s.t. copy+paste from the logfile gives a runnable command in the shell."""
- return " ".join([pipes.quote(arg) for arg in command])
+ return " ".join([shlex.quote(arg) for arg in command])
def unqualify(packages):
@@ -1833,6 +1834,45 @@ class Chroot:
return True
return False
+ def check_files_moved_usr(self, packages=[], files_before={}, files_after={}, warn_only=None):
+ """Check that no files were moved from /{bin|sbin|lib*} and /usr/{bin|sbin|lib*}"""
+
+ if settings.warn_on_usr_move == "disabled":
+ return
+
+ # For each path that is a file in each package, check that it did not move between
+ # /bin, /sbin or /lib* to the corresponding location under /usr, and viceversa.
+ # See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=994388#80
+ # If a file moved in a package that we are inspecting, print an error by default.
+ # Otherwise, print a warning.
+ broken = []
+ for old_path in files_before:
+ # '/' is a separate element in the parts list
+ old_path_parts = pathlib.Path(old_path).parts
+ if len(old_path_parts) < 3:
+ continue
+
+ if old_path_parts[1] == 'usr' and (old_path_parts[2] in ['bin', 'sbin'] or old_path_parts[2].startswith('lib')):
+ new_path = os.path.join("/", *old_path_parts[2:])
+ elif old_path_parts[1] in ['bin', 'sbin'] or old_path_parts[1].startswith('lib'):
+ new_path = "/usr" + old_path
+ else:
+ continue
+
+ # Skip over directories, multiple packages can ship files in the same directories
+ if new_path in files_after and os.path.isfile(self.relative(new_path)):
+ broken.append("%s %s => %s %s" % (old_path, files_before[old_path], new_path, files_after[new_path]))
+
+ if broken:
+ if settings.warn_on_usr_move == "warn" or warn_only:
+ logging.warning("WARN: File(s) moved between /{bin|sbin|lib*} and /usr/{bin|sbin|lib*}: %s" % indent_string("\n".join(broken)))
+ else:
+ logging.error("FAIL: File(s) moved between /{bin|sbin|lib*} and /usr/{bin|sbin|lib*}: %s" % indent_string("\n".join(broken)))
+ panic()
+ else:
+ logging.debug("No file moved between /{bin|sbin|lib*} and /usr/{bin|sbin|lib*}.")
+
+
def check_for_broken_symlinks(self, warn_only=None, file_owners={}):
"""Check that all symlinks in chroot are non-broken."""
if not settings.check_broken_symlinks:
@@ -2456,6 +2496,9 @@ def install_upgrade_test(chroot, chroot_state, package_files, packages, old_pack
chroot.check_for_no_processes()
chroot.check_for_broken_symlinks()
+ if settings.warn_on_usr_move != "disabled":
+ file_owners_before = chroot.get_files_owned_by_packages()
+
if settings.install_remove_install:
chroot.remove_packages(packages, ignore_errors=True)
@@ -2468,10 +2511,12 @@ def install_upgrade_test(chroot, chroot_state, package_files, packages, old_pack
chroot.disable_testdebs_repo()
- file_owners = chroot.get_files_owned_by_packages()
+ file_owners_after = chroot.get_files_owned_by_packages()
chroot.check_for_no_processes()
- chroot.check_for_broken_symlinks(file_owners=file_owners)
+ chroot.check_for_broken_symlinks(file_owners=file_owners_after)
+ if settings.warn_on_usr_move != "disabled":
+ chroot.check_files_moved_usr(packages, file_owners_before, file_owners_after)
# Remove all packages from the chroot that weren't there initially.
chroot.restore_selections(chroot_state, packages)
@@ -2479,9 +2524,9 @@ def install_upgrade_test(chroot, chroot_state, package_files, packages, old_pack
chroot.run_scripts("post_test")
chroot.check_for_no_processes(fail=True)
- chroot.check_for_broken_symlinks(file_owners=file_owners)
+ chroot.check_for_broken_symlinks(file_owners=file_owners_after)
- return check_results(chroot, chroot_state, file_owners)
+ return check_results(chroot, chroot_state, file_owners_after)
def save_meta_data(filename, chroot_state):
@@ -2610,6 +2655,9 @@ def install_and_upgrade_between_distros(package_files, packages_qualified):
chroot.check_for_no_processes()
+ if settings.warn_on_usr_move != "disabled":
+ file_owners_before = chroot.get_files_owned_by_packages()
+
os.environ["PIUPARTS_PHASE"] = "distupgrade"
chroot.upgrade_to_distros(settings.debian_distros[1:-1], distupgrade_packages, settings.upgrade_before_dist_upgrade)
@@ -2634,6 +2682,10 @@ def install_and_upgrade_between_distros(package_files, packages_qualified):
chroot.check_for_no_processes()
+ if settings.warn_on_usr_move != "disabled":
+ file_owners_after = chroot.get_files_owned_by_packages()
+ chroot.check_files_moved_usr(packages, files_before=file_owners_before, files_after=file_owners_after)
+
# Remove all packages from the chroot that weren't in the reference chroot.
chroot.restore_selections(chroot_state, packages_qualified)
@@ -3008,6 +3060,10 @@ def parse_command_line():
default=False,
help="Fail if broken symlinks are detected.")
+ parser.add_option("--warn-on-usr-move", action="store", default="disabled",
+ help="Whether to enable the test (with a warning or a failure) that checks if files are moved "
+ "between /{bin|sbin|lib*} and /usr/{bin|sbin|lib*}. Accepted values: 'disabled' (default), 'warn', 'fail'.")
+
parser.add_option("--log-level", action="store", metavar='LEVEL',
default="dump",
help="Displays messages from LEVEL level, possible values are: error, info, dump, debug. The default is dump.")
@@ -3103,6 +3159,7 @@ def parse_command_line():
settings.warn_on_debsums_errors = opts.warn_on_debsums_errors
settings.warn_on_install_over_symlink = opts.warn_on_install_over_symlink
settings.warn_if_inadequate = not opts.fail_if_inadequate
+ settings.warn_on_usr_move = opts.warn_on_usr_move
settings.pedantic_purge_test = opts.pedantic_purge_test
settings.ignored_files += opts.ignore
settings.ignored_patterns += opts.ignore_regex
@@ -3138,6 +3195,10 @@ def parse_command_line():
logging.error("Scripts directory is not a directory: %s" % sdir)
panic()
+ if settings.warn_on_usr_move not in ["disabled", "warn", "fail"]:
+ logging.error("--warn-on-usr-move must be one of 'disabled', 'warn', 'fail'")
+ panic()
+
if not settings.debian_distros:
settings.debian_distros = defaults.get_distribution()
=====================================
tests/test_piuparts.py
=====================================
@@ -1,63 +1,69 @@
-import unittest
-from mox3 import mox
import os
import shutil
+import unittest
+from unittest.mock import patch
+
import piuparts
from piuparts import is_broken_symlink
class DefaultsFactoryTests(unittest.TestCase):
-
def setUp(self):
- self.mox = mox.Mox()
self.df = piuparts.DefaultsFactory()
piuparts.settings = piuparts.Settings()
- def tearDown(self):
- self.mox.UnsetStubs()
-
def test_new_defaults_return_debian_defaults(self):
# mock the guess_flavor function as it runs lsb_release in a subprocess
- self.mox.StubOutWithMock(self.df, 'guess_flavor')
- self.df.guess_flavor().AndReturn('debian')
- self.mox.ReplayAll()
-
- defaults = self.df.new_defaults()
- self.mox.VerifyAll()
-
- self.assertEqual(defaults.get_keyring(), '/usr/share/keyrings/debian-archive-keyring.gpg')
+ with patch.object(
+ self.df, "guess_flavor", return_value="debian"
+ ) as guess_flavor_mock:
+ defaults = self.df.new_defaults()
+ guess_flavor_mock.assert_called_once()
+
+ self.assertEqual(
+ defaults.get_keyring(), "/usr/share/keyrings/debian-archive-keyring.gpg"
+ )
self.assertEqual(defaults.get_components(), ["main", "contrib", "non-free"])
- self.assertEqual(defaults.get_mirror(), [("http://deb.debian.org/debian", ["main", "contrib", "non-free"])])
- self.assertEqual(defaults.get_distribution(), ['sid'])
+ self.assertEqual(
+ defaults.get_mirror(),
+ [("http://deb.debian.org/debian", ["main", "contrib", "non-free"])],
+ )
+ self.assertEqual(defaults.get_distribution(), ["sid"])
def test_new_defaults_return_ubuntu_defaults(self):
- # mock the guess_flavor function as it runs lsb_release in a subprocess
- self.mox.StubOutWithMock(self.df, 'guess_flavor')
- self.df.guess_flavor().AndReturn('ubuntu')
- self.mox.ReplayAll()
-
- defaults = self.df.new_defaults()
- self.mox.VerifyAll()
-
- self.assertEqual(defaults.get_keyring(), '/usr/share/keyrings/ubuntu-archive-keyring.gpg')
- self.assertEqual(defaults.get_components(), ["main", "universe", "restricted", "multiverse"])
- self.assertEqual(defaults.get_mirror(), [("http://archive.ubuntu.com/ubuntu", ["main", "universe", "restricted", "multiverse"])])
+ with patch.object(
+ self.df, "guess_flavor", return_value="ubuntu"
+ ) as guess_flavor_mock:
+ defaults = self.df.new_defaults()
+ guess_flavor_mock.assert_called_once()
+
+ self.assertEqual(
+ defaults.get_keyring(), "/usr/share/keyrings/ubuntu-archive-keyring.gpg"
+ )
+ self.assertEqual(
+ defaults.get_components(), ["main", "universe", "restricted", "multiverse"]
+ )
+ self.assertEqual(
+ defaults.get_mirror(),
+ [
+ (
+ "http://archive.ubuntu.com/ubuntu",
+ ["main", "universe", "restricted", "multiverse"],
+ )
+ ],
+ )
def test_new_defaults_panics_with_unknown_flavor(self):
- # mock the guess_flavor function as it runs lsb_release in a subprocess
- # and the panic function as it would use sys.exit()
- self.mox.StubOutWithMock(self.df, 'guess_flavor')
- self.df.guess_flavor().AndReturn('centos')
- self.mox.StubOutWithMock(piuparts, 'panic')
- piuparts.panic().AndReturn('Oh dear! Its CentOS!')
- self.mox.ReplayAll()
-
- defaults = self.df.new_defaults()
- self.mox.VerifyAll()
-
- # panic() would cause sys.exit() so no Defaults object would
- # ever be returned
- self.assertEqual(defaults, None)
+ with patch.object(
+ self.df, "guess_flavor", return_value="centos"
+ ) as guess_flavor_mock, patch.object(
+ piuparts, "panic", side_effect=SystemExit
+ ) as panic_mock:
+ with self.assertRaises(SystemExit):
+ self.df.new_defaults()
+
+ guess_flavor_mock.assert_called_once()
+ panic_mock.assert_called_once()
class IsBrokenSymlinkTests(unittest.TestCase):
View it on GitLab: https://salsa.debian.org/debian/piuparts/-/compare/e3be3151eac85582a032b68870044be84116a0f3...fbca117e15ad105e1560abd1fdc77e63864f00e5
--
View it on GitLab: https://salsa.debian.org/debian/piuparts/-/compare/e3be3151eac85582a032b68870044be84116a0f3...fbca117e15ad105e1560abd1fdc77e63864f00e5
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/piuparts-devel/attachments/20230111/a8b19b4d/attachment-0001.htm>
More information about the Piuparts-devel
mailing list