[Piuparts-commits] [SCM] piuparts git repository branch, develop, updated. 0.49-117-g7e3aa5d
Andreas Beckmann
anbe at debian.org
Sun Mar 3 09:33:12 UTC 2013
The following commit has been merged in the develop branch:
commit d7d3d0c227a18595de524fafc5bb2d27a90c5dfb
Author: Andreas Beckmann <anbe at debian.org>
Date: Sat Mar 2 12:18:56 2013 +0100
p.conf: add [global] proxy setting
if set, the http_proxy environment variable will be set to this value
by piuparts-master/-slave/-report (and will be propagated to piuparts)
overriding any http_proxy setting
Signed-off-by: Andreas Beckmann <anbe at debian.org>
diff --git a/README.txt b/README.txt
index 1a3d640..6c9a02e 100644
--- a/README.txt
+++ b/README.txt
@@ -494,6 +494,16 @@ used for all further sections.
after load drops below 'slave-load-max - 1.0'. Floating point
value. Defaults to 0 (= disabled).
+* "proxy" sets the http_proxy that will be used for fetching
+ Packages files etc. (by master/slave/report) and .debs etc. (by
+ piuparts). This will override a http_proxy setting in the
+ environment. By default (no value being set) the http_proxy
+ variable from the environment will be used (and no proxy will be
+ used if this is not set). It is highly recommended to use a proxy
+ running on localhost (e.g. installing squid and using a setting of
+ "http://localhost:3128") due to the high bandwidth consumption of
+ piuparts and repeated downloading of the same files.
+
==== section specific configuration
The section specific settings will be reloaded each time a section
diff --git a/README_server.txt b/README_server.txt
index d9b5a13..3a6e8d0 100644
--- a/README_server.txt
+++ b/README_server.txt
@@ -7,7 +7,8 @@ piuparts-slave (which ships that file), too, or use the template
'sections' to be tested (e.g. 'sid') and define references to the Debian
mirror. Note that the server can place a significant load on the
repository. Consider setting up a local mirror, or a caching proxy for http
-and apt-get, to reduce the load.
+and apt-get, to reduce the load. Running multiple slaves on a fast host can
+easily saturate a 100 MBit link.
Edit /etc/sudoers.d/piuparts to grant permissions to the piupartss user.
Start the server using /usr/sbin/piuparts_slave_run, which will launch a
diff --git a/TODO b/TODO
index bd5ced8..cc1a7ca 100644
--- a/TODO
+++ b/TODO
@@ -141,11 +141,6 @@ http://docs.python.org/library/logging.html
- move shell cronjobs functionality into master, slave & report
-- document that it's highly recommended to run a proxy or mirror on localhost -
- running multiple slaves on a fast host can easily saturate a 100 Mbit link :-)
-- there could be a config option that sets appropriate configuration for master,
- slave, and (especially) piuparts
-
1.0 should really have automated testing of piuparts itself...
-----------------------------------------------------------------
diff --git a/debian/changelog b/debian/changelog
index c3b15fb..b5ba7a8 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -15,6 +15,7 @@ piuparts (0.50) UNRELEASED; urgency=low
overwritten files owned by other packages as failures.
* piuparts.conf:
- New global settings:
+ + proxy
+ slave-directory
+ known-problem-directory (for known_problem definitions)
+ backup-directory (for the master script prepare_backup)
@@ -32,6 +33,7 @@ piuparts (0.50) UNRELEASED; urgency=low
- Optimize reserve() and skip unavailable candidates.
* piuparts-master.py:
- Sort packages to be tested by importance, i.e. descending waiting_count.
+ - Use piuparts.conf global proxy setting as http_proxy.
- Use distros.conf to compute URLs.
* piuparts-slave.py:
- Tarball recreation can be disabled by setting max-tgz-age to 0.
@@ -43,11 +45,13 @@ piuparts (0.50) UNRELEASED; urgency=low
(piuparts.conf global setting slave-load-max) is exceeded. Operation
will be resumed after load drops below 'slave-load-max - 1.0'. Disabled
by default.
+ - Use piuparts.conf global proxy setting as http_proxy.
- Use distros.conf to compute URLs.
* piuparts-analyze.py:
- Add support for magic "$DISTRO/None" versions.
* piuparts-report.py:
- Call r.dev_off() after generating a plot. (Closes: #657799)
+ - Use piuparts.conf global proxy setting as http_proxy.
- Use distros.conf to compute URLs.
* Makefile:
- Add DESTDIR support.
diff --git a/piuparts-master.py b/piuparts-master.py
index 0ca0e08..f643962 100644
--- a/piuparts-master.py
+++ b/piuparts-master.py
@@ -57,6 +57,7 @@ class Config(piupartslib.conf.Config):
{
"log-file": None,
"master-directory": ".",
+ "proxy": None,
"mirror": None,
"distro": None,
"area": None,
@@ -305,6 +306,8 @@ def main():
if len(sys.argv) == 2:
global_config = Config(section="global")
global_config.read(CONFIG_FILE)
+ if global_config["proxy"]:
+ os.environ["http_proxy"] = global_config["proxy"]
master_directory = global_config["master-directory"]
section = sys.argv[1]
diff --git a/piuparts-report.py b/piuparts-report.py
index e62bbeb..bcad34e 100644
--- a/piuparts-report.py
+++ b/piuparts-report.py
@@ -397,6 +397,7 @@ class Config(piupartslib.conf.Config):
"output-directory": "html",
"master-directory": ".",
"description": "",
+ "proxy": None,
"mirror": None,
"distro": None,
"area": None,
@@ -1294,6 +1295,8 @@ def main():
global_config = Config(section="global")
global_config.read(CONFIG_FILE)
+ if global_config["proxy"]:
+ os.environ["http_proxy"] = global_config["proxy"]
section_names = global_config["sections"].split()
master_directory = global_config["master-directory"]
output_directory = global_config["output-directory"]
diff --git a/piuparts-slave.py b/piuparts-slave.py
index edb77e1..f3353ce 100644
--- a/piuparts-slave.py
+++ b/piuparts-slave.py
@@ -77,6 +77,7 @@ class Config(piupartslib.conf.Config):
"master-directory": ".",
"master-command": None,
"log-file": "piuparts-master.log",
+ "proxy": None,
"mirror": None,
"piuparts-command": "sudo piuparts",
"piuparts-flags": "",
@@ -855,6 +856,8 @@ def main():
section_names = []
global_config = Config(section="global")
global_config.read(CONFIG_FILE)
+ if global_config["proxy"]:
+ os.environ["http_proxy"] = global_config["proxy"]
if len(sys.argv) > 1:
section_names = sys.argv[1:]
else:
--
piuparts git repository
More information about the Piuparts-commits
mailing list