[Piuparts-devel] Multiple/remote piuparts slaves

[Andreas Beckmann]

On 2012-03-04 05:12, Dave Steele wrote:
> Is anyone else running multiple slaves? If so, what performance/issues do
> you see?

I'm running multiple slaves but on the host system, i.e. one piupartss
user, one shared slave/ directory. All slaves process different
sections. I added some locking to piuparts-slave to pfevent concurrent
processing of a section which would be racy. So they won't run the
master for the same section concurrently. I should really push these
patches somewhere :-)
The master needs to do something similar to avoid the race you noticed.
it can just return "busy" instead of hello and the slave will retry this
section later (with my patches ...) instead of failing.

With the huge amount of differnt tests I run, piuparts-report takes
quite some time and I seem to get I/O bound (or at least sync-bound) on
a quad-core machine with 8 GB ram running piuparts-report (uses 4+ GB
RAM at the end) and 3-4 slaves. Slaves use TMPDIR on tmpfs.
I just changed the HDD which was constantly running full and probably
put a lot of pressure on the filesystem as well, right now it seems to
work well and piuparts-report is back to 2.5 hours (was sometimes >8).

Andreas