[Piuparts-devel] [Git][debian/piuparts][master] 32 commits: piuparts-report.py: fix link to states.png
Holger Levsen
gitlab at salsa.debian.org
Tue Jan 19 15:22:56 GMT 2021
Holger Levsen pushed to branch master at Debian / piuparts
Commits:
81ba3455 by Ansgar at 2019-09-10T22:54:18+02:00
piuparts-report.py: fix link to states.png
- - - - -
68f2fab7 by Stuart Prescott at 2019-12-06T00:16:11+11:00
Add timeout when fetching packages files
- - - - -
3f3a475d by Nicolas Dandrimont at 2020-01-17T16:04:41+01:00
Merge branch 'stuart/piuparts-tmp/develop/timeout' into develop
See merge request https://salsa.debian.org/debian/piuparts/merge_requests/15
- - - - -
af510fd3 by Nicolas Dandrimont at 2020-01-17T16:09:20+01:00
Merge branch 'ansgar/piuparts-fix-link-to-states.png' into HEAD
See https://salsa.debian.org/debian/piuparts/merge_requests/10
- - - - -
a78a2db3 by Nis Martensen at 2020-01-26T10:58:44+01:00
piuparts-analyze: unbreak sorting of version lists
fixes:
ERROR processing fail/<package>.log
<class 'TypeError'>
- - - - -
a1023d21 by Nicolas Dandrimont at 2020-02-12T23:14:31+01:00
Merge branch 'nma-guest/piuparts-analyze_no_cmp' into HEAD
- - - - -
86c88a5c by Nicolas Dandrimont at 2020-02-13T01:44:20+01:00
piuparts-analyze: print full exceptions when failing
- - - - -
18bb611c by Nicolas Dandrimont at 2020-02-13T01:44:26+01:00
piuparts-analyze: fetch all bugs in bulk instead of package by package
The BTS SOAP interface struggles when we do a lot of package-based queries; We
even get throttled (and get ConnectionErrors) when doing the queries for a usual
piuparts-analyze run on pejacevic.
Querying the BTS by bug numbers instead doesn't hit the throttle, even when
getting all bugs usertagged for piuparts. This bulk query makes the full
piuparts-analyze run go way faster.
- - - - -
0bfbe61d by Nicolas Dandrimont at 2020-02-13T01:44:26+01:00
piuparts-analyze: only wait for 60 seconds when a given section is busy
With the performance improvement of querying the BTS in bulk, piuparts-analyze
now spends most of its time waiting for busy sections. Let's wait a bit less.
- - - - -
b6123494 by Nicolas Dandrimont at 2020-03-13T11:18:51+01:00
Add exemption for /var/lib/dpkg/cmethopt
This file, which tracks the current dselect download method, is now handled by
dselect (>= 1.20.0~), but it is also created by debootstrap while it is doing
its apt/dselect setup.
Closes: #953756.
- - - - -
c00105c5 by Nicolas Dandrimont at 2020-03-13T11:28:37+01:00
Add myself to uploaders
- - - - -
f5322e22 by Nicolas Dandrimont at 2020-03-13T11:30:27+01:00
Update d/changelog
- - - - -
2ebb2ba2 by Nicolas Dandrimont at 2020-03-13T13:34:10+01:00
d/rules: Add workaround for dh-dwz bug #933541.
- - - - -
a72a8242 by Nicolas Dandrimont at 2020-03-20T22:37:35+01:00
Set Description last when generating a dummy package
Having Depends last confuses apt 2.0.0.
Reported-By: Ferenc Wágner (https://alioth-lists.debian.net/pipermail/piuparts-devel/2020-March/009069.html)
- - - - -
4f81106e by Julien Cristau at 2020-04-09T10:24:13+02:00
Make the summary generation atomic
- - - - -
093fc7c2 by Nicolas Dandrimont at 2020-04-09T08:42:16+00:00
Merge branch 'atomic-summary' into 'develop'
Make the summary generation atomic
See merge request debian/piuparts!28
- - - - -
a29095e7 by Nicolas Dandrimont at 2020-04-22T10:13:08+02:00
Start piuparts-slave automatically after reboots again
- - - - -
8a313fbd by Nicolas Dandrimont at 2020-04-29T22:29:55+02:00
Add piu-slave-conova-01
- - - - -
418151e2 by Nicolas Dandrimont at 2020-04-29T22:38:07+02:00
Add news entry for piu-slave-conova-01
- - - - -
d7155993 by Nicolas Dandrimont at 2020-04-29T22:54:18+02:00
Move piu-slave-ubc-01 to deb.d.o
- - - - -
9fb9f46a by Nis Martensen at 2021-01-18T19:49:19+01:00
autopkgtest: add a must-fail test
The existing must-succeed test protects against potential bugs that make
piuparts always fail. Other potential bugs might make piuparts always
succeed, so let's add a must-fail test to cover this case, too.
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
0dc80c16 by Nis Martensen at 2021-01-18T19:50:02+01:00
d/p-[ms].lintian-overrides: update to python3
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
3e42799d by Holger Levsen at 2021-01-18T19:51:23+01:00
Bump standards version to 4.5.1, no changes needed.
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
7d99442a by Nis Martensen at 2021-01-18T19:54:21+01:00
piuparts.py: run: do not crash on non-utf-8 command output
This should fix the crash reported in
https://alioth-lists.debian.net/pipermail/piuparts-devel/2020-September/009165.html
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
d7835aeb by Nis Martensen at 2021-01-18T19:56:15+01:00
piuparts.py: Prevent KeyError in diff_meta_data
Since commit c39004577d42c0c6ec63266df3ec2f86b03b8653 different dicts
were used for iterating and modifying, so we never modify the dicts we
are currently iterating over. However, the commit was incomplete because
it was longer guaranteed that each key is still part of the dict we are
trying to modify, but no checks were added.
The result are possible crashes such as the one reported in 968842:
Traceback (most recent call last):
File "/usr/sbin/piuparts", line 3267, in <module>
main()
File "/usr/sbin/piuparts", line 3252, in main
process_packages(package_list)
File "/usr/sbin/piuparts", line 3173, in process_packages
if not install_purge_test(chroot, chroot_state,
File "/usr/sbin/piuparts", line 2414, in install_purge_test
return check_results(chroot, chroot_state, file_owners, deps_info=deps_info)
File "/usr/sbin/piuparts", line 2235, in check_results
(new, removed, modified) = diff_meta_data(reference_info, current_info)
File "/usr/sbin/piuparts", line 2041, in diff_meta_data
del tree2_c[name]
KeyError: '/var/spool/news/in.coming/tmp/'
Such KeyErrors can be prevented by checking if a key exists in the dict
before trying to drop it. Adding explicit checks is simpler than
adding a try..except clause here because there are already if clauses
that we can adapt.
Closes: #968842
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
7a07ebd9 by Holger Levsen at 2021-01-18T20:11:18+01:00
Fix day-of-week for changelog entry 0.16-1.
Changes-By: lintian-brush
Fixes: lintian: debian-changelog-has-wrong-day-of-week
See-also: https://lintian.debian.org/tags/debian-changelog-has-wrong-day-of-week.html
- - - - -
e7bc5b35 by Holger Levsen at 2021-01-18T20:21:57+01:00
add d/changelog entries for recent commits
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
9abb0e66 by Holger Levsen at 2021-01-18T20:22:43+01:00
d/rules: drop workaround for #933541 now that the bug is fixed
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
4207dd31 by Holger Levsen at 2021-01-19T01:37:55+01:00
Revert "d/rules: drop workaround for #933541 now that the bug is fixed"
This reverts commit 9abb0e661fcc0456cbf40ed62a25cd65b27bab56.
- - - - -
9ece1d47 by Holger Levsen at 2021-01-19T01:40:40+01:00
explain why this workaround should be kept for the time being
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
504b1fc1 by Holger Levsen at 2021-01-19T16:15:44+01:00
Merge branch 'master' into develop
- - - - -
31862192 by Holger Levsen at 2021-01-19T16:20:50+01:00
release as 1.1.2
Signed-off-by: Holger Levsen <holger at layer-acht.org>
- - - - -
16 changed files:
- conf/crontab-slave.in
- debian/changelog
- debian/control
- debian/piuparts-master.lintian-overrides
- debian/piuparts-slave.lintian-overrides
- debian/rules
- debian/tests/smoke-test
- htdocs/index.tpl
- htdocs/news.tpl
- instances/Makefile
- + instances/piuparts.conf-header.piu-slave-conova-01
- piuparts-analyze.py
- piuparts-report.py
- piuparts.py
- piupartslib/__init__.py
- piupartslib/pkgsummary.py
Changes:
=====================================
conf/crontab-slave.in
=====================================
@@ -6,7 +6,8 @@ PATH=@sharedir@/piuparts/slave:/usr/bin:/bin
#
# start slave_run automatically after reboots
#
-#@reboot slave_cleanup ; slave_run
+ at reboot slave_cleanup ; slave_run
+
#42 * * * * slave_cleanup ; ! slave_run >/dev/null 2>&1 || echo "piuparts-slave started by cron"
#
=====================================
debian/changelog
=====================================
@@ -1,3 +1,46 @@
+piuparts (1.1.2) unstable; urgency=medium
+
+ * Team upload.
+
+ [ Nis Martensen ]
+ * piuparts.py:
+ - prevent KeyError in diff_meta_data. (Closes: #968842)
+ - do not crash on non-utf-8 command output.
+ * piuparts-analyze: unbreak sorting of version lists.
+ * autopkgtest: add a must-fail test.
+ * d/p-[ms].lintian-overrides: update to python3/
+
+ [ Nicolas Dandrimont ]
+ * Add exemption for /var/lib/dpkg/cmethopt. (Closes: #953756)
+ * piuparts-analyze:
+ - print full exceptions when failing.
+ - fetch all bugs in bulk instead of package by package.
+ - only wait for 60 seconds when a given section is busy.
+ * Add myself to uploaders.
+ * d/rules: Add workaround for dh-dwz bug #933541.
+ * Set Description last when generating a dummy package.
+ * Start piuparts-slave automatically after reboots again.
+ * Add piu-slave-conova-01.
+ * Add news entry for piu-slave-conova-01.
+ * Move piu-slave-ubc-01 to deb.d.o.
+
+ [ Ansgar ]
+ * piuparts-report.py: fix link to states.png.
+
+ [ Stuart Prescott ]
+ * Add timeout when fetching packages files.
+
+ [ Julien Cristau ]
+ * Make the summary generation atomic.
+
+ [ Holger Levsen ]
+ * Bump standards version to 4.5.1, no changes needed.
+
+ [ Debian Janitor ]
+ * Fix day-of-week for changelog entry 0.16-1.
+
+ -- Holger Levsen <holger at debian.org> Tue, 19 Jan 2021 16:17:25 +0100
+
piuparts (1.1.1) unstable; urgency=medium
* Team upload to unstuck python-debianbts migration.
@@ -3398,7 +3441,7 @@ piuparts (0.16-1) unstable; urgency=low
* debian/control: Made the dependency on debootstrap be versioned.
Closes: #355875.
- -- Lars Wirzenius <liw at iki.fi> Fri, 22 Mar 2006 22:40:00 +0200
+ -- Lars Wirzenius <liw at iki.fi> Wed, 22 Mar 2006 22:40:00 +0200
piuparts (0.15-1) unstable; urgency=low
=====================================
debian/control
=====================================
@@ -1,8 +1,9 @@
Source: piuparts
Maintainer: piuparts developers team <piuparts-devel at alioth-lists.debian.net>
+Uploaders: Nicolas Dandrimont <olasd at debian.org>
Section: devel
Priority: optional
-Standards-Version: 4.4.1
+Standards-Version: 4.5.1
Rules-Requires-Root: no
Build-Depends:
debhelper-compat (= 12),
=====================================
debian/piuparts-master.lintian-overrides
=====================================
@@ -1,5 +1,5 @@
# It's ok, because it depends on piuparts-common that has the dependency..
-piuparts-master: python-script-but-no-python-dep usr/share/piuparts/master/detect_well_known_errors #!/usr/bin/python3
-piuparts-master: python-script-but-no-python-dep usr/share/piuparts/piuparts-analyze #!/usr/bin/python3
-piuparts-master: python-script-but-no-python-dep usr/share/piuparts/piuparts-master-backend #!/usr/bin/python3
-piuparts-master: python-script-but-no-python-dep usr/share/piuparts/piuparts-report #!/usr/bin/python3
+piuparts-master: python3-script-but-no-python3-dep usr/share/piuparts/master/detect_well_known_errors #!/usr/bin/python3
+piuparts-master: python3-script-but-no-python3-dep usr/share/piuparts/piuparts-analyze #!/usr/bin/python3
+piuparts-master: python3-script-but-no-python3-dep usr/share/piuparts/piuparts-master-backend #!/usr/bin/python3
+piuparts-master: python3-script-but-no-python3-dep usr/share/piuparts/piuparts-report #!/usr/bin/python3
=====================================
debian/piuparts-slave.lintian-overrides
=====================================
@@ -1,2 +1,2 @@
# It's ok, because it depends on piuparts-common that has the dependency..
-piuparts-slave: python-script-but-no-python-dep usr/share/piuparts/piuparts-slave #!/usr/bin/python3
+piuparts-slave: python3-script-but-no-python3-dep usr/share/piuparts/piuparts-slave #!/usr/bin/python3
=====================================
debian/rules
=====================================
@@ -17,3 +17,8 @@ override_dh_auto_install:
override_dh_python3:
dh_python3 -p piuparts-master -p piuparts-slave /usr/share/piuparts --shebang=/usr/bin/python3
dh_python3 -N piuparts-master -N piuparts-slave
+
+override_dh_dwz:
+ # Workaround for https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=933541
+ # (this workaround can be dropped with debhelper from buster-backports)
+ :
=====================================
debian/tests/smoke-test
=====================================
@@ -16,6 +16,9 @@ WORKDIR=$(mktemp -d)
trap "rm -rf $WORKDIR" 0 INT QUIT ABRT PIPE TERM
cd $WORKDIR
+
+# set up a very simple test package
+
mkdir -p t/DEBIAN t/usr t/etc
cat >t/DEBIAN/control <<EOF
Package: t
@@ -30,4 +33,35 @@ EOF
dpkg-deb -b t
+# this should always succeed
test_this piuparts t.deb
+
+
+# another simple package, but set up for failure
+
+mkdir -p f/DEBIAN f/usr f/etc
+cat >f/DEBIAN/control <<EOF
+Package: f
+Version: 4
+Maintainer: Piu Parts <piuparts-devel at alioth-lists.debian.net>
+Priority: optional
+Architecture: all
+Installed-Size: 0
+Description: Auto Package Test Fail Dummy
+ Extremely simple binary package for piuparts testing - fail version
+EOF
+
+cat >f/DEBIAN/postinst <<EOF
+#! /bin/sh
+mkdir -p /etc/f/
+touch /etc/f/ailure
+EOF
+
+chmod +x f/DEBIAN/postinst
+
+dpkg-deb -b f
+
+# it is an error if this succeeds
+test_this piuparts f.deb && false
+
+exit 0
=====================================
htdocs/index.tpl
=====================================
@@ -43,10 +43,11 @@
</tr>
<tr class="normalrow">
<td class="contentcell2">
- Since 2005 piuparts is run systematically on all packages in Debian, at first only on <a href="/sid" target="_blank">unstable</a>. Today piuparts.debian.org is testing many suites by running piuparts in <a href="/doc/README_server.html" target="_blank">master/slave mode</a>. This setup currently consists of two hosts: <a href="https://db.debian.org/machines.cgi?host=pejacevic" target="_blank">pejacevic.debian.org</a> and and <a href="https://db.debian.org/machines.cgi?host=piu-slave-ubc-01" target="_blank">piu-slave-ubc-01.debian.org</a>:
+ Since 2005 piuparts is run systematically on all packages in Debian, at first only on <a href="/sid" target="_blank">unstable</a>. Today piuparts.debian.org is testing many suites by running piuparts in <a href="/doc/README_server.html" target="_blank">master/slave mode</a>. This setup currently consists of three hosts: <a href="https://db.debian.org/machines.cgi?host=pejacevic" target="_blank">pejacevic.debian.org</a>, <a href="https://db.debian.org/machines.cgi?host=piu-slave-ubc-01" target="_blank">piu-slave-ubc-01.debian.org</a> and <a href="https://db.debian.org/machines.cgi?host=piu-slave-conova-01" target="_blank">piu-slave-conova-01.debian.org</a>:
<ul>
- <li> pejacevic acts as the piuparts-master, which is responsible for scheduling test jobs to the slaves. The other main task is to generate the reports which are served via https://piuparts.debian.org.</li>
- <li> piu-slave-ubc-01 runs four piuparts-slave instances, which then run piuparts itself.</li>
+ <li> pejacevic acts as the piuparts-master, which is responsible for scheduling test jobs to the slaves. The other main task is to generate the reports which are served via https://piuparts.debian.org;</li>
+ <li> piu-slave-ubc-01 runs four piuparts-slave instances, which then run piuparts itself;</li>
+ <li> piu-slave-conova-01 also runs four piuparts-slave instances, which then run piuparts itself.</li>
</ul>
</tr>
<tr class="normalrow">
=====================================
htdocs/news.tpl
=====================================
@@ -4,6 +4,12 @@
News
</td>
</tr>
+ <tr class="normalrow">
+ <td class="contentcell2">
+ <b>2020-04-29</b>
+ Thanks to DSA, a new worker host, <a href="https://db.debian.org/machines.cgi?host=piu-slave-conova-01">piu-slave-conova-01</a>, has been introduced. It will run 4 parallel worker processes on the amd64 arch.
+ </td>
+ </tr>
<tr class="normalrow">
<td class="contentcell2">
<b>2020-01-08</b>
=====================================
instances/Makefile
=====================================
@@ -3,11 +3,11 @@ MIRROR = http://deb.debian.org/debian/
ARCH_piu-slave-1und1-01 = i386
MIRROR_piu-slave-1und1-01 = http://mirror.1und1.de/debian/
-MIRROR_piu-slave-ubc-01 = http://mirror-ubc.debian.org/debian/
GENCONF = piuparts.conf.pejacevic
GENCONF += piuparts.conf.piu-slave-1und1-01
GENCONF += piuparts.conf.piu-slave-ubc-01
+GENCONF += piuparts.conf.piu-slave-conova-01
all: $(GENCONF)
=====================================
instances/piuparts.conf-header.piu-slave-conova-01
=====================================
@@ -0,0 +1,8 @@
+#
+# This is the configuration file for piuparts running in master-slave mode on
+# - piu-slave-conova-01.debian.org (as slave).
+# Except for the mirror configuration this file is identical to the one
+# for pejacevic and piu-slave-ubc-01.
+#
+# For more information on this setup see https://salsa.debian.org/debian/piuparts/blob/develop/README_pejacevic.txt
+#
=====================================
piuparts-analyze.py
=====================================
@@ -4,6 +4,7 @@
# Copyright 2005 Lars Wirzenius (liw at iki.fi)
# Copyright 2011 Mika Pflüger (debian at mikapflueger.de)
# Copyright © 2012-2017 Andreas Beckmann (anbe at debian.org)
+# Copyright © 2020 Nicolas Dandrimont (nicolas at dandrimont.eu)
#
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
@@ -31,18 +32,19 @@ move the failed log to ./bugged as well.
"""
from __future__ import print_function
+from collections import defaultdict
+import fcntl
import os
import sys
import time
import re
import shutil
-import subprocess
-import fcntl
+import traceback
import debianbts
import apt_pkg
-from signal import alarm, signal, SIGALRM
from collections import deque
+from functools import cmp_to_key
import piupartslib.conf
from piupartslib.conf import MissingSection
@@ -57,78 +59,71 @@ chroot_pattern = re.compile(r"tmp/tmp.*?'")
############################################################################
-class BTS_Timeout(Exception):
- pass
-
-
-def alarm_handler(signum, frame):
- raise BTS_Timeout
-
-
class PiupartsBTS():
def __init__(self):
- self._bugs_usertagged_piuparts = None
- self._bugs_in_package = dict()
- self._bugs_affecting_package = dict()
- self._bug_versions = dict()
+ self._fetched = False
+ self._bugs = {}
+ self._bugs_in_package = defaultdict(set)
+ self._bugs_affecting_package = defaultdict(set)
+ self._bug_versions = {}
self._queries = 0
- self._misses = 0
- def all_piuparts_bugs(self):
- if self._bugs_usertagged_piuparts is None:
- self._bugs_usertagged_piuparts = debianbts.get_usertag("debian-qa at lists.debian.org", ['piuparts'])['piuparts']
- return self._bugs_usertagged_piuparts
+ def fetch_bug_data(self):
+ if self._fetched:
+ return
+ self._fetched = True
+ bug_nums = debianbts.get_usertag("debian-qa at lists.debian.org", ['piuparts'])['piuparts']
+ bugs = debianbts.get_status(bug_nums)
+ for bug in bugs:
+ bug_num = bug.bug_num
+
+ self._bugs[bug_num] = bug
+
+ # Populate bug number -> versions map
+ versions = []
+ for version in bug.found_versions:
+ # debianbts returns found versions in the format package/1.2.3 or 1.2.3 which will become 1.2.3
+ v = version.rsplit('/', 1)[-1]
+ if v == 'None':
+ continue
+ versions.append(v)
+ if versions:
+ versions.sort(key=cmp_to_key(apt_pkg.version_compare))
+ self._bug_versions[bug_num] = versions
+
+ # Populate package -> bug map
+ for package in bug.package.split(','):
+ if package.startswith('src:'):
+ package = package[4:]
+ self._bugs_in_package[package].add(bug_num)
+
+ # Populate affected package -> bug map
+ for package in bug.affects:
+ if package.startswith('src:'):
+ package = package[4:]
+ self._bugs_affecting_package[package].add(bug_num)
def bugs_in(self, package):
- if not package in self._bugs_in_package:
- self._misses += 1
- signal(SIGALRM, alarm_handler)
- alarm(120)
- bugs = debianbts.get_bugs(package=package, bugs=self.all_piuparts_bugs(), archive='both')
- bugs += debianbts.get_bugs(package='src:' + package, bugs=self.all_piuparts_bugs(), archive='both')
- alarm(0)
- self._bugs_in_package[package] = sorted(set(bugs), reverse=True)
self._queries += 1
+ self.fetch_bug_data()
return self._bugs_in_package[package]
def bugs_affecting(self, package):
- if not package in self._bugs_affecting_package:
- self._misses += 1
- signal(SIGALRM, alarm_handler)
- alarm(120)
- bugs = debianbts.get_bugs(affects=package, bugs=self.all_piuparts_bugs(), archive='both')
- bugs += debianbts.get_bugs(affects='src:' + package, bugs=self.all_piuparts_bugs(), archive='both')
- alarm(0)
- self._bugs_affecting_package[package] = sorted(set(bugs), reverse=True)
self._queries += 1
+ self.fetch_bug_data()
return self._bugs_affecting_package[package]
def bug_versions(self, bug):
"""Gets a list of only the version numbers for which the bug is found.
Newest versions are returned first."""
- # debianbts returns it in the format package/1.2.3 or 1.2.3 which will become 1.2.3
- if not bug in self._bug_versions:
- self._misses += 1
- signal(SIGALRM, alarm_handler)
- alarm(60)
- found_versions = debianbts.get_status([bug])[0].found_versions
- alarm(0)
- versions = []
- for found_version in found_versions:
- v = found_version.rsplit('/', 1)[-1]
- if v == "None":
- # ignore $DISTRO/None versions
- pass
- else:
- versions.append(v)
- self._bug_versions[bug] = list(reversed(sorted(versions, cmp=apt_pkg.version_compare))) or ['~']
self._queries += 1
- return self._bug_versions[bug]
+ self.fetch_bug_data()
+ return self._bug_versions.get(bug, ['~'])
def print_stats(self):
- print("PiupartsBTS: %d queries, %d forwarded to debianbts" % (self._queries, self._misses))
+ print("PiupartsBTS: %d queries" % self._queries)
piupartsbts = PiupartsBTS()
@@ -271,7 +266,8 @@ def mark_logs_with_reported_bugs():
moved = False
abugs = piupartsbts.bugs_affecting(pname)
bugs = piupartsbts.bugs_in(pname)
- for bug in abugs + bugs:
+ all_bugs = sorted(abugs | bugs, reverse=True)
+ for bug in all_bugs:
if moved:
break
if bug in abugs:
@@ -306,13 +302,12 @@ def mark_logs_with_reported_bugs():
bts_update_found(bug, pversion)
break
if not moved:
- write_bug_file(failed_log, abugs + bugs)
+ write_bug_file(failed_log, all_bugs)
except KeyboardInterrupt:
raise
- except:
+ except Exception:
print('ERROR processing %s' % failed_log)
- print(sys.exc_info()[0])
- alarm(0)
+ traceback.print_exc()
def main():
@@ -359,7 +354,7 @@ def main():
os.chdir(oldcwd)
except Busy:
print("Section is busy")
- todo.append((section_name, time.time() + 300))
+ todo.append((section_name, time.time() + 60))
except MissingSection as e:
print("Configuration Error in section '%s': %s" % (section_name, e))
print("")
=====================================
piuparts-report.py
=====================================
@@ -761,7 +761,7 @@ def create_section_navigation(section_names, current_section, doc_root):
% (doc_root, current_section)
tablerows += "<tr><td class=\"contentcell\"><a href=\"%s/%s/source/\">by source package</a></td></tr>\n" \
% (doc_root, current_section)
- tablerows += "<tr><td class=\"contentcell\">states <a href=\"%s/%s/states.png/\">graph</a></td></tr>\n" \
+ tablerows += "<tr><td class=\"contentcell\">states <a href=\"%s/%s/states.png\">graph</a></td></tr>\n" \
% (doc_root, current_section)
tablerows += "<tr class=\"titlerow\"><td class=\"alerttitlecell\">all tested suites</td></tr>"
for section in section_names:
=====================================
piuparts.py
=====================================
@@ -297,6 +297,7 @@ class Settings:
"/var/lib/dpkg/arch",
"/var/lib/dpkg/available",
"/var/lib/dpkg/available-old",
+ "/var/lib/dpkg/cmethopt", # Bug#953756
"/var/lib/dpkg/diversions",
"/var/lib/dpkg/diversions-old",
"/var/lib/dpkg/lock",
@@ -558,7 +559,7 @@ def run(command, ignore_errors=False, timeout=0):
devnull = open('/dev/null', 'r')
p = subprocess.Popen(command, env=env, stdin=devnull,
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
- universal_newlines=True)
+ universal_newlines=True, errors="backslashreplace")
output = ""
excessive_output = False
if timeout > 0:
@@ -655,15 +656,16 @@ def make_metapackage(name, depends, conflicts, arch='all'):
control['Architecture'] = arch
control['Maintainer'] = ('piuparts developers team '
'<piuparts-devel at alioth-lists.debian.net>')
- control['Description'] = ('Dummy package to satisfy dependencies - '
- 'created by piuparts\n'
- ' This package was created automatically by '
- 'piuparts and can safely be removed')
if depends:
control['Depends'] = depends
if conflicts:
control['Conflicts'] = conflicts
+ control['Description'] = ('Dummy package to satisfy dependencies - '
+ 'created by piuparts\n'
+ ' This package was created automatically by '
+ 'piuparts and can safely be removed')
+
create_file(os.path.join(tmpdir, name, 'DEBIAN', 'control'),
control.dump())
@@ -2011,11 +2013,11 @@ def diff_meta_data(tree1, tree2, quiet=False):
name = name[1:]
else:
verbose = False
- if name in tree1:
+ if name in tree1_c:
if verbose:
logging.info("IGNORED PATH at 1: %s" % name)
del tree1_c[name]
- if name in tree2:
+ if name in tree2_c:
if verbose:
logging.info("IGNORED PATH at 2: %s" % name)
del tree2_c[name]
@@ -2029,20 +2031,20 @@ def diff_meta_data(tree1, tree2, quiet=False):
pat = re.compile(pattern)
for name in tree1.keys():
m = pat.search(name)
- if m:
+ if m and name in tree1_c:
if verbose:
logging.info("IGNORED PATH at 1: %s" % name)
del tree1_c[name]
for name in tree2.keys():
m = pat.search(name)
- if m:
+ if m and name in tree2_c:
if verbose:
logging.info("IGNORED PATH at 2: %s" % name)
del tree2_c[name]
modified = []
for name in tree1.keys():
- if name in tree2_c:
+ if name in tree1_c and name in tree2_c:
if objects_are_different(tree1[name], tree2[name]):
if not quiet:
logging.debug("Modified(user, group, mode, size, target): %s expected%s != found%s" %
=====================================
piupartslib/__init__.py
=====================================
@@ -93,7 +93,7 @@ def open_packages_url(url):
error = None
for ext in ['.xz', '.bz2', '.gz', '']:
try:
- socket = urllib.request.urlopen(url + ext)
+ socket = urllib.request.urlopen(url + ext, timeout=30)
except urllib.error.HTTPError as e:
error = e
else:
=====================================
piupartslib/pkgsummary.py
=====================================
@@ -83,6 +83,7 @@ from __future__ import print_function
import json
import datetime
from collections import namedtuple, defaultdict
+import os
import six
@@ -221,8 +222,9 @@ def tooltip(summary, pkg):
def write_summary(summary, fname):
- with open(fname, 'w') as fl:
+ with open(fname + '.tmp', 'w') as fl:
json.dump(summary, fl, sort_keys=True, indent=1)
+ os.rename(fname + '.tmp', fname)
def read_summary(fname):
View it on GitLab: https://salsa.debian.org/debian/piuparts/-/compare/b07a73830b7ef13ba02c2300566f53205da112d1...31862192384ea68ad5d490c6fbf99f6f78f11314
--
View it on GitLab: https://salsa.debian.org/debian/piuparts/-/compare/b07a73830b7ef13ba02c2300566f53205da112d1...31862192384ea68ad5d490c6fbf99f6f78f11314
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/piuparts-devel/attachments/20210119/b3fcf92f/attachment-0001.html>
More information about the Piuparts-devel
mailing list