[Piuparts-commits] rev 299 - piatti piatti/home/piupartsm piatti/home/piupartsm/bin piatti/home/piupartss piatti/home/piupartss/bin piatti/org/piuparts.debian.org/etc piatti/org/piuparts.debian.org/htdocs trunk trunk/debian

Holger Levsen holger at alioth.debian.org
Sun Mar 22 12:43:23 UTC 2009 

Author: holger
Date: 2009-03-22 12:43:22 +0000 (Sun, 22 Mar 2009)
New Revision: 299

Added:
   piatti/org/piuparts.debian.org/etc/piuparts.conf.goldwasser
   piatti/org/piuparts.debian.org/etc/piuparts.conf.piatti
Removed:
   piatti/home/piupartsm/.bashrc
   piatti/home/piupartss/.bashrc
   piatti/org/piuparts.debian.org/etc/piuparts.conf
Modified:
   piatti/README.txt
   piatti/home/piupartsm/bin/detect_network_issues
   piatti/home/piupartsm/crontab
   piatti/home/piupartss/bin/update-piuparts-setup
   piatti/org/piuparts.debian.org/htdocs/style.css
   trunk/Makefile
   trunk/README.txt
   trunk/TODO
   trunk/debian/changelog
   trunk/debian/control
   trunk/debian/rules
   trunk/piuparts-master.py
   trunk/piuparts-report.py
   trunk/piuparts-slave.py
   trunk/piuparts.conf.sample
Log:
    - improve layout, generate navigation for the pages.
  * Break backwards compatibility of the configuration files for master-slave-
    mode. Merge those three into one: /etc/piuparts/piuparts/piuparts.conf.
    Introduce a new global section in piuparts.conf which is shared among the
    other sections there.
  * Makefile: 
    - add "~$date" to versionstring if building an unreleased version, thus
      adding dpkg-dev to build-depends.
    - split install target into install-doc, install-conf and install, to aid
      development and deployment from trunk.



Modified: piatti/README.txt
===================================================================
--- piatti/README.txt	2009-03-20 12:38:20 UTC (rev 298)
+++ piatti/README.txt	2009-03-22 12:43:22 UTC (rev 299)
@@ -4,7 +4,6 @@
 todo
 ----
 - script for starting piuparts-slave in screen and cronjob to send mail with its output
-- cronjob to check number of mounts on /org/piuparts.d.o/tmp/
 - use local mirror
 - there should be a 2nd group of piuparts-people. those who can sudo into piupartsm to process logfiles. maybe make that the qa-group
 
@@ -28,8 +27,10 @@
 
 piuparts installation from svn source
 -------------------------------------
-* sudo apt-get install apt python debootstrap lsof lsb-release python-debian 
-* Copy 'svn:/svn.debian.org/svn/piuparts/piatti/home/piupartss/bin/update-piuparts-setup' on the host and run it. It assumes you want to set it up in '/org/piuparts.debian.org' and does all further svn checkouts as well as source code installation. It needs the piupartss and piupartsm user set up as described below, though.
+* sudo apt-get install apt python debootstrap lsof lsb-release python-debian make dpkg-dev docbook2x python-support docbook-xml asciidoc 
+* you need a webserver too, if you run the master
+* Copy 'svn://svn.debian.org/svn/piuparts/piatti/home/piupartss/bin/update-piuparts-setup' on the host and run it. It assumes you want to set it up in '/org/piuparts.debian.org' and does all further svn checkouts as well as source code installation. It needs the piupartss and piupartsm user set up as described below, though.
+* sudo ln -s /org/piuparts.debian.org/etc/ /etc/piuparts
 	
 Apache configuration
 --------------------
@@ -44,7 +45,7 @@
         ErrorLog /var/log/apache2/piuparts.debian.org-error.log
         CustomLog /var/log/apache2/piuparts.debian.org-access.log combined
 
-        DocumentRoot /srv/piuparts.debian.org/htdocs
+        DocumentRoot /org/piuparts.debian.org/htdocs
 
         HostnameLookups Off
         UseCanonicalName Off
@@ -59,10 +60,14 @@
 ----------------------------------
 'piuparts-report' is run from '~piupartsm/crontab'
 
+
 Cronjobs to aid problem spotting
 --------------------------------
-'~piupartsm/bin/detect_network_issues' is run daily from cron and should detect failed piuparts runs due to network issues on the host.
+Reside in '~piupartsm/bin/' and are run by '~piupartsm/crontab'.
 
+- 'detect_network_issues' should detect failed piuparts runs due to network issues on the host.
+- 'detect_stale_mounts' should detect stale mountpoints (usually of /proc) from failed piuparts runs.
+
 More checks should be added as we become aware of them.
 
 
@@ -70,7 +75,7 @@
 ----------------------------------------------
 ----
 piupartss at piatti:/org/piuparts.debian.org$ ./update-piuparts-setup
-piupartss at piatti:/org/piuparts.debian.org/slave$ nice python ../../share/piuparts/piuparts-slave 
+piupartss at piatti:/org/piuparts.debian.org/slave$ nice python ../share/piuparts/piuparts-slave 
 ----
 
 filing bugs

Deleted: piatti/home/piupartsm/.bashrc

Modified: piatti/home/piupartsm/bin/detect_network_issues
===================================================================
--- piatti/home/piupartsm/bin/detect_network_issues	2009-03-20 12:38:20 UTC (rev 298)
+++ piatti/home/piupartsm/bin/detect_network_issues	2009-03-22 12:43:22 UTC (rev 299)
@@ -1,5 +1,21 @@
 #!/bin/sh
 
+# Copyright 2009 Holger Levsen (holger at layer-acht.org)
+# 
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 2 of the License, or (at your
+# option) any later version.
+# 
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
+# Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
 MASTER=/org/piuparts.debian.org/master
 DISTROS="sid squeeze"
 

Modified: piatti/home/piupartsm/crontab
===================================================================
--- piatti/home/piupartsm/crontab	2009-03-20 12:38:20 UTC (rev 298)
+++ piatti/home/piupartsm/crontab	2009-03-22 12:43:22 UTC (rev 299)
@@ -10,4 +10,5 @@
 #
 42 6 * * * PYTHONPATH=/org/piuparts.debian.org/lib/python2.4/site-packages:/org/piuparts.debian.org/lib/python2.5/site-packages nice python /org/piuparts.debian.org/share/piuparts/piuparts-report
 
-42 7 * * * /home/piupartsm/bin/detect_network_issues
+42 7 * * * /home/piupartsm/bin/detect_stale_mounts
+43 7 * * * /home/piupartsm/bin/detect_network_issues

Deleted: piatti/home/piupartss/.bashrc

Modified: piatti/home/piupartss/bin/update-piuparts-setup
===================================================================
--- piatti/home/piupartss/bin/update-piuparts-setup	2009-03-20 12:38:20 UTC (rev 298)
+++ piatti/home/piupartss/bin/update-piuparts-setup	2009-03-22 12:43:22 UTC (rev 299)
@@ -3,10 +3,23 @@
 #
 #  update piuparts setup on piatti.debian.org
 #
-#   (c) 2009 Holger Levsen         holger at layer-acht.org
-#   GPL2 licenced
-#
 
+# Copyright 2009 Holger Levsen (holger at layer-acht.org)
+# 
+# This program is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by the
+# Free Software Foundation; either version 2 of the License, or (at your
+# option) any later version.
+# 
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
+# Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+
 if [ "`id -n -u`" != "piupartss" ] ; then
         echo please run this script as piupartss user
         exit 1
@@ -24,6 +37,10 @@
 pwd
 svn up
 svn status
+grep -q PYTHONPATH .bashrc || {
+	echo PYTHONPATH=/org/piuparts.debian.org/lib/python2.4/site-packages:/org/piuparts.debian.org/lib/python2.5/site-packages >> .bashrc
+	echo export PYTHONPATH >> .bashrc
+}
 
 #
 # update master home, that's mostly crontab to run piuparts-reports
@@ -37,7 +54,12 @@
 fi
 sudo su - piupartsm -c "pwd ; svn up ; svn status"
 sudo su - piupartsm -c "crontab crontab"
+grep -q PYTHONPATH ~piupartsm/.bashrc || {
+	echo PYTHONPATH=/org/piuparts.debian.org/lib/python2.4/site-packages:/org/piuparts.debian.org/lib/python2.5/site-packages >> ~piupartsm/.bashrc
+	echo export PYTHONPATH >> ~piupartsm/.bashrc
+}
 
+
 #
 # update /org/piuparts.debian.org
 #
@@ -54,8 +76,11 @@
 svn status
 mkdir -p master slave
 sudo chown piupartss.piuparts slave
-sudo chown piupartsm.piuparts master
-sudo chmod 755 master slave
+sudo chown piupartsm.piuparts master htdocs
+sudo chmod 755 master slave htdocs
+cd /org/piuparts.debian.org/etc
+# to support multiple host with this setup
+ln -sf piuparts.conf.$HOSTNAME piuparts.conf
 
 #
 # update source
@@ -71,5 +96,10 @@
 pwd
 svn up
 svn status
-sudo make prefix=/org/piuparts.debian.org install-py 
+sudo make prefix=/org/piuparts.debian.org docdir=/org/piuparts.debian.org/htdocs/doc/ install install-doc
 
+#
+# create working dir
+#
+sudo mkdir -p /org/piuparts.debian.org/tmp
+

Deleted: piatti/org/piuparts.debian.org/etc/piuparts.conf

Copied: piatti/org/piuparts.debian.org/etc/piuparts.conf.goldwasser (from rev 297, piatti/org/piuparts.debian.org/etc/piuparts.conf)
===================================================================
--- piatti/org/piuparts.debian.org/etc/piuparts.conf.goldwasser	                        (rev 0)
+++ piatti/org/piuparts.debian.org/etc/piuparts.conf.goldwasser	2009-03-22 12:43:22 UTC (rev 299)
@@ -0,0 +1,39 @@
+#
+# This is the configuration file for piuparts running in master-slave mode.
+#
+#
+
+[global]
+sections = sid squeeze
+master-host = localhost
+master-user = piupartsm
+master-directory = /org/piuparts.debian.org/master/
+output-directory = /org/piuparts.debian.org/htdocs/
+idle-sleep = 300
+
+[sid]
+description = Debian sid / main, installation and removal tests
+master-command = PYTHONPATH=/org/piuparts.debian.org/lib/python2.4/site-packages:/org/piuparts.debian.org/lib/python2.5/site-packages python /org/piuparts.debian.org/share/piuparts/piuparts-master sid
+packages-url = http://ftp.de.debian.org/debian/dists/sid/main/binary-amd64/Packages.bz2
+mirror = http://ftp.de.debian.org/debian/
+piuparts-cmd = sudo /org/piuparts.debian.org/sbin/piuparts --no-symlinks --scriptsdir /etc/piuparts/scripts/ --tmpdir /org/piuparts.debian.org/tmp
+distro = sid
+chroot-tgz = sid.tar.gz
+upgrade-test-distros = 
+upgrade-test-chroot-tgz = 
+max-reserved = 30
+log-file = sid-master.log
+
+[squeeze]
+description = Debian squeeze / main, installation and removal tests as well as installation in lenny and upgrade to squeze and then removal tests
+master-command = PYTHONPATH=/org/piuparts.debian.org/lib/python2.4/site-packages:/org/piuparts.debian.org/lib/python2.5/site-packages python /org/piuparts.debian.org/share/piuparts/piuparts-master squeeze
+packages-url = http://ftp.de.debian.org/debian/dists/squeeze/main/binary-amd64/Packages.bz2
+mirror = http://ftp.de.debian.org/debian/
+piuparts-cmd = sudo /org/piuparts.debian.org/sbin/piuparts --no-symlinks --scriptsdir /etc/piuparts/scripts/ --tmpdir /org/piuparts.debian.org/tmp
+distro = squeeze
+chroot-tgz = squeeze.tar.gz
+upgrade-test-distros = lenny squeeze
+upgrade-test-chroot-tgz = lenny.tar.gz
+max-reserved = 15
+log-file = squeeze-master.log
+

Copied: piatti/org/piuparts.debian.org/etc/piuparts.conf.piatti (from rev 297, piatti/org/piuparts.debian.org/etc/piuparts.conf)
===================================================================
--- piatti/org/piuparts.debian.org/etc/piuparts.conf.piatti	                        (rev 0)
+++ piatti/org/piuparts.debian.org/etc/piuparts.conf.piatti	2009-03-22 12:43:22 UTC (rev 299)
@@ -0,0 +1,39 @@
+#
+# This is the configuration file for piuparts running in master-slave mode.
+#
+#
+
+[global]
+sections = sid squeeze
+master-host = localhost
+master-user = piupartsm
+master-directory = /org/piuparts.debian.org/master/
+output-directory = /org/piuparts.debian.org/htdocs/
+idle-sleep = 300
+
+[sid]
+description = Debian sid / main, installation and removal tests
+master-command = PYTHONPATH=/org/piuparts.debian.org/lib/python2.4/site-packages:/org/piuparts.debian.org/lib/python2.5/site-packages python /org/piuparts.debian.org/share/piuparts/piuparts-master sid
+packages-url = http://ftp.fi.debian.org/debian/dists/sid/main/binary-amd64/Packages.bz2
+mirror = http://ftp.fi.debian.org/debian/
+piuparts-cmd = sudo /org/piuparts.debian.org/sbin/piuparts --no-symlinks --scriptsdir /etc/piuparts/scripts/ --tmpdir /org/piuparts.debian.org/tmp
+distro = sid
+chroot-tgz = sid.tar.gz
+upgrade-test-distros = 
+upgrade-test-chroot-tgz = 
+max-reserved = 100
+log-file = sid-master.log
+
+[squeeze]
+description = Debian squeeze / main, installation and removal tests as well as installation in lenny and upgrade to squeze and then removal tests
+master-command = PYTHONPATH=/org/piuparts.debian.org/lib/python2.4/site-packages:/org/piuparts.debian.org/lib/python2.5/site-packages python /org/piuparts.debian.org/share/piuparts/piuparts-master squeeze
+packages-url = http://ftp.fi.debian.org/debian/dists/squeeze/main/binary-amd64/Packages.bz2
+mirror = http://ftp.fi.debian.org/debian/
+piuparts-cmd = sudo /org/piuparts.debian.org/sbin/piuparts --no-symlinks --scriptsdir /etc/piuparts/scripts/ --tmpdir /org/piuparts.debian.org/tmp
+distro = squeeze
+chroot-tgz = squeeze.tar.gz
+upgrade-test-distros = lenny squeeze
+upgrade-test-chroot-tgz = lenny.tar.gz
+max-reserved = 50
+log-file = squeeze-master.log
+

Modified: piatti/org/piuparts.debian.org/htdocs/style.css
===================================================================
--- piatti/org/piuparts.debian.org/htdocs/style.css	2009-03-20 12:38:20 UTC (rev 298)
+++ piatti/org/piuparts.debian.org/htdocs/style.css	2009-03-22 12:43:22 UTC (rev 299)
@@ -218,7 +218,6 @@
 
 /* Tables */
 table.containertable {
-    width: 100%;
     clear: both;
     background-color: #ddd;
     border: none; 

Modified: trunk/Makefile
===================================================================
--- trunk/Makefile	2009-03-20 12:38:20 UTC (rev 298)
+++ trunk/Makefile	2009-03-22 12:43:22 UTC (rev 299)
@@ -4,6 +4,7 @@
 mandir = $(sharedir)/man
 man1dir = $(mandir)/man1
 libdir = $(prefix)/lib
+docdir = $(prefix)/share/doc/piuparts/
 site24 = $(libdir)/python2.4/site-packages
 site25 = $(libdir)/python2.5/site-packages
 etcdir = $(prefix)/etc
@@ -16,39 +17,40 @@
 
 ignore = -I fdmount -N
 
-all: piuparts-doc install-conf install-py
+all: install-conf install-doc install
 
-piuparts-doc:
+install-doc:
 	docbook2x-man --encoding=utf-8 piuparts.docbook
 	asciidoc -a toc -a toclevels=3 README.txt
 	a2x -a toc -a toclevels=3 -f pdf README.txt
 	rm README.xml
+	install -d $(docdir)/
+	for file in README.txt README.html README.pdf; do \
+	    install -m 0755 $$file $(docdir)/ ; done
+	
+	install -d $(man1dir) 
+	install -m 0644 piuparts.1 $(man1dir)
+	gzip -9f $(man1dir)/piuparts.1
 
 install-conf:
 	install -d $(etcdir)/piuparts
 	install -m 0644 piuparts.conf.sample $(etcdir)/piuparts/piuparts.conf
 
-install-py: 
+install:
 	install -d $(sbindir) 
 	echo $(version)
 	sed -e 's/__PIUPARTS_VERSION__/$(version)/g' piuparts.py > piuparts
 	install piuparts $(sbindir)/piuparts
-
-	install -d $(man1dir) 
-	install -m 0644 piuparts.1 $(man1dir)
-	gzip -9f $(man1dir)/piuparts.1
-
+	
 	install -d $(sharedir)/piuparts
 	for file in piuparts-slave piuparts-master piuparts-report piuparts-analyze; do \
 	    install -m 0755 $$file.py $(sharedir)/piuparts/$$file ; done
-
+	
 	install -d $(site24)/piupartslib
 	install -d $(site25)/piupartslib
 	install -m 0644 piupartslib/*.py $(site24)/piupartslib
 	install -m 0644 piupartslib/*.py $(site25)/piupartslib
 
-install: all
-
 check:
 	python piuparts.py unittest
 	python unittests.py

Modified: trunk/README.txt
===================================================================
--- trunk/README.txt	2009-03-20 12:38:20 UTC (rev 298)
+++ trunk/README.txt	2009-03-22 12:43:22 UTC (rev 299)
@@ -71,7 +71,7 @@
 create this tarball with the command (again, as root):
 
 ---- 
-    # puilder create
+    # pbuilder create
 ---- 
  
 then you only have to remember to update this tarball with:

Modified: trunk/TODO
===================================================================
--- trunk/TODO	2009-03-20 12:38:20 UTC (rev 298)
+++ trunk/TODO	2009-03-22 12:43:22 UTC (rev 299)
@@ -3,24 +3,25 @@
 
 must for 0.36
 
-- globalize more config variables.
-- document this in README
-- document sections for piuparts-slave in README. 
-- the sections in the index page generated by piuparts-report.py are hardcoded atm..
-  (fix by merging configfiles, also sections should have 
-   desctription="Debian sid, consisting of main contrib non-free, testing installation and removal in sid"
-- the templates used by update-reports.py should be taken from /etc/piuparts/templates/
-  and not be included in the python source
+- document globalised config variables this in README
+- the sections in the nav bar are hardcoded atm..
 - rename packages.txt into $section_done_today.txt and include sections
 - create $section_packages.txt with into about all packages 
 - better header (link to FAQ, point out to read the bottom first..) for piuparts-logs
 - debug is on, even though its off in the config
+- master.log is hardcoded in p-master.py?!
 - take care of old conf files on upgrades
 - take care of old pyc files from python-central
 - scripts in /usr/share/piuparts/ probably a FHS violation
+- make sure README.* gets included in the package
+- rm piuparts.css?
+- the templates used by update-reports.py should be taken from /etc/piuparts/templates/
+  and not be included in the python source
 
 for 0.37 and on
 
+- publish FAI classes to setup piuparts.$fqdn automatically
+
 - upgrade-reports should have a list of available arch and list packages 
   only available on untested archs in a new state "(depends-)not-available-on-tested-archs"
 
@@ -29,6 +30,8 @@
 - master should (per default) only schedule packages which are not available 
   on the master arch to slaves of different archs -> "schedule-evenly-to-slaves = no"
 
+- master.log grows to fast and there is no mechanism to stop it
+
 - piuparts can't currently test upgrades of required packages. (Because they
   cannot be removed, it assumes these are untestable, which is only true for
   removal tests...

Modified: trunk/debian/changelog
===================================================================
--- trunk/debian/changelog	2009-03-20 12:38:20 UTC (rev 298)
+++ trunk/debian/changelog	2009-03-22 12:43:22 UTC (rev 299)
@@ -21,8 +21,6 @@
     - assume packages in state "circular-dependency" are testable. True, they
       are not testable without the circular dependent package, so in case of 
       failure analysing will be harder.
-  * Break backwards compatibility of the configuration files for master-slave-
-    mode. Merge those three into one: /etc/piuparts/piuparts/piuparts.conf
   * Install scripts for master-slave mode without .py extension.
   * piuparts-slave.py:
     - support looping trough multiple sections in slave.conf via new config 
@@ -36,9 +34,17 @@
     - support looping trough multiple sections in report.conf via new config
       value "sections". 
     - provide links to logfiles in statistics page.
-    - improve look.
+    - improve layout, generate navigation for the pages.
+  * Break backwards compatibility of the configuration files for master-slave-
+    mode. Merge those three into one: /etc/piuparts/piuparts/piuparts.conf.
+    Introduce a new global section in piuparts.conf which is shared among the
+    other sections there.
   * Always use alphabetic time zone abbreviation in timestamps.
-  * Makefile: add "~$date" to versionstring if building an unreleased version.
+  * Makefile: 
+    - add "~$date" to versionstring if building an unreleased version, thus
+      adding dpkg-dev to build-depends.
+    - split install target into install-doc, install-conf and install, to aid
+      development and deployment from trunk.
   * debian/control: depend on python (>>2.4), make dependency to python-debian
     unversioned and add build-dependencies for debhelper and asciidoc.
   * Rewrite debian/rules using debhelper.

Modified: trunk/debian/control
===================================================================
--- trunk/debian/control	2009-03-20 12:38:20 UTC (rev 298)
+++ trunk/debian/control	2009-03-22 12:43:22 UTC (rev 299)
@@ -10,7 +10,7 @@
 Section: devel
 Priority: extra
 Standards-Version: 3.8.0
-Build-Depends: debhelper (>=5.0.42)
+Build-Depends: debhelper (>=5.0.42), dpkg-dev
 Build-Depends-Indep: docbook2x, python-support, docbook-xml, asciidoc
 Vcs-Svn: svn://svn.debian.org/svn/piuparts/trunk/
 Vcs-Browser: http://svn.debian.org/wsvn/piuparts/trunk/

Modified: trunk/debian/rules
===================================================================
--- trunk/debian/rules	2009-03-20 12:38:20 UTC (rev 298)
+++ trunk/debian/rules	2009-03-22 12:43:22 UTC (rev 299)
@@ -17,7 +17,7 @@
 	dh_clean -k
 	dh_installdirs
 	dh_installman
-	$(MAKE) prefix=$(CURDIR)/debian/piuparts/usr etcdir=$(CURDIR)/debian/piuparts/etc install
+	$(MAKE) prefix=$(CURDIR)/debian/piuparts/usr etcdir=$(CURDIR)/debian/piuparts/etc all
 
 binary-arch:
 binary-indep: build install

Modified: trunk/piuparts-master.py
===================================================================
--- trunk/piuparts-master.py	2009-03-20 12:38:20 UTC (rev 298)
+++ trunk/piuparts-master.py	2009-03-22 12:43:22 UTC (rev 299)
@@ -233,7 +233,7 @@
 
     def write_summaries(self):
         self.write_counts_summary()
-        self.write_packages_summary()
+        #self.write_packages_summary()
 
 
 def main():

Modified: trunk/piuparts-report.py
===================================================================
--- trunk/piuparts-report.py	2009-03-20 12:38:20 UTC (rev 298)
+++ trunk/piuparts-report.py	2009-03-22 12:43:22 UTC (rev 299)
@@ -55,7 +55,7 @@
      <img src="http://piuparts.debian.org/images/debian.png" border="0" hspace="0" vspace="0" alt="Debian Project"></a>
     Quality Assurance
    </h1>
-  <table class="reddy" width="97%">
+  <table class="reddy">
    <tr>
     <td class="reddy">
      <img src="http://piuparts.debian.org/images/red-upperleft.png" align="left" border="0" hspace="0" vspace="0"
@@ -162,7 +162,7 @@
    <table class="righttable">
     <tr class="titlerow">
      <td class="titlecell">
-      $title
+      $title in $section
      </td>
     </tr>
     <tr class="normalrow">
@@ -190,7 +190,7 @@
    <table class="righttable">
     <tr class="titlerow">
      <td class="titlecell">
-      Packages in state "$state"
+      Packages in state "$state" in $section
      </td>
     </tr>
     <tr class="normalrow">
@@ -208,9 +208,18 @@
    <table class="righttable">
     <tr class="titlerow">
      <td class="titlecell" colspan="3">
-      Statistics of packages per section
+      $section statistics
      </td>
     </tr>
+     <tr class="normalrow">
+     <td class="contentcell2" colspan="3">
+      $description
+     </td>
+    <tr class="titlerow">
+     <td class="titlecell" colspan="3">
+      Packages per state
+     </td>
+    </tr>
     $tablerows
     <tr class="titlerow">
      <td class="titlecell" colspan="3">
@@ -323,11 +332,10 @@
             {
                 "sections": "report",
                 "output-directory": "html",
-                "index-page": "index.html",
                 "packages-url": None,
                 "master-directory": ".",
-            },
-            ["output-directory"])
+                "description": "",
+            }, "")
 
 
 def setup_logging(log_level, log_file_name):
@@ -386,6 +394,7 @@
     f.write(htmlpage.safe_substitute( {
                 "time": time.strftime("%Y-%m-%d %H:%M %Z"),
                 "title": html_protect(title),
+                "section": html_protect(self._config.section),
                 "preface": preface,
                 "count": len(packages),
                 "versioncount": version_count,
@@ -451,16 +460,17 @@
     def __init__(self, section):
         self._config = Config(section=section)
         self._config.read(CONFIG_FILE)
-        self._output_directory = os.path.abspath(os.path.join(self._config["output-directory"], self._config.section))
-        if not os.path.exists(self._output_directory):
-            os.mkdir(self._output_directory)
-        self._master_directory = os.path.abspath(os.path.join(self._config["master-directory"], self._config.section))
 
-    def output(self):
+    def output(self, master_directory, output_directory):
         logging.debug("-------------------------------------------")
         logging.debug("Running section " + self._config.section)
+        self._master_directory = os.path.abspath(os.path.join(master_directory, self._config.section))
         if os.path.exists(self._master_directory):
 
+            self._output_directory = os.path.abspath(os.path.join(output_directory, self._config.section))
+            if not os.path.exists(self._output_directory):
+                os.mkdir(self._output_directory)
+
             oldcwd = os.getcwd()
             os.chdir(self._master_directory)
 
@@ -506,6 +516,8 @@
             htmlpage = string.Template(HTML_HEADER + SECTION_STATS_BODY_TEMPLATE + HTML_FOOTER)
             write_file(os.path.join(self._output_directory, "index.html"), htmlpage.safe_substitute( {
                 "time": time.strftime("%Y-%m-%d %H:%M %Z"),
+                "section": html_protect(self._config.section),
+                "description": html_protect(self._config["description"]),
                 "tablerows": tablerows,
                 "packagesurl": html_protect(self._config["packages-url"]), 
                }))
@@ -529,12 +541,12 @@
                 write_file(os.path.join(self._output_directory, "state-%s.html" % state), htmlpage.safe_substitute( {
                                             "time": time.strftime("%Y-%m-%d %H:%M %Z"),
                                             "state": html_protect(state),
+                                            "section": html_protect(self._config.section),
                                             "list": list
                                            }))
 
             os.chdir(oldcwd)
 
-
 def main():
     setup_logging(logging.DEBUG, None)
 
@@ -552,12 +564,12 @@
     sections = []
     for section_name in section_names:
         section = Section(section_name)
-        section.output()
+        section.output(master_directory=global_config["master-directory"],output_directory=global_config["output-directory"])
         sections.append(section)
 
     logging.debug("Writing index page")
     htmlpage = string.Template(HTML_HEADER + INDEX_BODY_TEMPLATE + HTML_FOOTER)
-    write_file(global_config["index-page"], htmlpage.safe_substitute( {
+    write_file(os.path.join(global_config["output-directory"],"index.html"), htmlpage.safe_substitute( {
                                  "time": time.strftime("%Y-%m-%d %H:%M %Z"),
                               }))
 

Modified: trunk/piuparts-slave.py
===================================================================
--- trunk/piuparts-slave.py	2009-03-20 12:38:20 UTC (rev 298)
+++ trunk/piuparts-slave.py	2009-03-22 12:43:22 UTC (rev 299)
@@ -68,7 +68,7 @@
                 "master-directory": ".",
                 "master-command": None,
                 "mirror": None,
-                "piuparts-cmd": "python piuparts.py",
+                "piuparts-cmd": "sudo piuparts",
                 "distro": "sid",
                 "chroot-tgz": None,
                 "upgrade-test-distros": None,
@@ -77,8 +77,7 @@
                 "debug": "no",
                 "keep-sources-list": "no",
                 "arch": None,
-            },
-            ["master-host", "master-user", "master-command"])
+            }, "")
 
 
 class MasterNotOK(Exception):
@@ -221,7 +220,7 @@
         if not os.path.exists(self._slave_directory):
             os.mkdir(self._slave_directory)
 
-    def setup(self):
+    def setup(self, master_host, master_user, master_directory):
         if self._config["debug"] in ["yes", "true"]:
             self._logger = logging.getLogger()
             self._logger.setLevel(logging.DEBUG)
@@ -243,9 +242,9 @@
                 os.mkdir(dir)
     
         self._slave = Slave()
-        self._slave.set_master_host(self._config["master-host"])
-        self._slave.set_master_user(self._config["master-user"])
-        self._slave.set_master_directory(self._config["master-directory"])
+        self._slave.set_master_host(master_host)
+        self._slave.set_master_user(master_user)
+        self._slave.set_master_directory(master_directory)
         self._slave.set_master_command(self._config["master-command"])
     
         for dir in ["pass", "fail", "untestable", "reserved"]:
@@ -254,7 +253,7 @@
                 os.makedirs(dir)
         os.chdir(oldcwd)
 
-    def run(self):
+    def run(self, idle_sleep):
         logging.info("-------------------------------------------")
         logging.info("Running section " + self._config.section)
         self._slave.connect_to_master()
@@ -278,7 +277,7 @@
 
         if not self._slave.get_reserved():
             logging.debug("Nothing to do, sleeping for a bit")
-            time.sleep(int(self._config["idle-sleep"]))
+            time.sleep(int(idle_sleep))
         else:
             packages_files = {}
             distros = [self._config["distro"]] + self._config["upgrade-test-distros"].split()
@@ -428,12 +427,12 @@
     sections = []
     for section_name in section_names:
         section = Section(section_name)
-        section.setup()
+        section.setup(master_host=global_config["master-host"],master_user=global_config["master-user"],master_directory=global_config["master-directory"])
         sections.append(section)
 
     while True:
         for section in sections:
-            section.run()
+            section.run(idle_sleep=global_config["idle-sleep"])
 
 
 if __name__ == "__main__":

Modified: trunk/piuparts.conf.sample
===================================================================
--- trunk/piuparts.conf.sample	2009-03-20 12:38:20 UTC (rev 298)
+++ trunk/piuparts.conf.sample	2009-03-22 12:43:22 UTC (rev 299)
@@ -4,36 +4,37 @@
 # The sample file does not set all required values, to prevent the
 # master from being run accidentally.
 # 
-# You MUST make sure that the packages url is set correctly. The sample
-# file does not set it at all, to prevent the master from being run
-# accidentally.
+# You MUST make sure that master-host, master-user, master-directory, mirror
+# and packages-url are also set correctly. The sample file does not set it at
+# all, to prevent the master from being run accidentally.
 #
 #
 
 [global]
 sections = sid
-
-[sid]
-sections = sid
-log-file = slave.log
-idle-sleep = 300
 # master-host = localhost
 # master-user = piuparts
 # master-directory = .
 output-directory = html
-master-command = python /usr/share/piuparts/piuparts-master
+idle-sleep = 300
+
+[sid]
+description = "Debian sid / main"
+master-command = python /usr/share/piuparts/piuparts-master sid
 # mirror = http://your.mirror.here/debian
 # packages-url = http://your.mirror.here/debian/dists/sid/main/binary-i386/Packages.bz2
 piuparts-cmd = sudo piuparts
+# another example:
+# piuparts-cmd = sudo piuparts --no-symlinks --scriptsdir /etc/piuparts/scripts/ --tmpdir /var/tmp
 distro = sid
 chroot-tgz = sid.tar.gz
 upgrade-test-distros = 
 upgrade-test-chroot-tgz = 
 max-reserved = 50
 debug = no
-og-file = /dev/null
+log-file = sid-master.log
 
-
+## another example:
 ## [s-p-u-i386]
 ## master-command = python /usr/share/piuparts/piuparts-master.py s-p-u-i386
 ## packages-url = http://ftp.debian.org/debian/dists/stable-proposed-updates/binary-i386/Packages.bz2

More information about the Piuparts-commits mailing list