[Piuparts-devel] _throttle_if_overloaded stability

[Dave Steele]

in p-s: check load frequently and sleep if it exceeds threshold

as part of the current for-holger:

+
+    def _throttle_if_overloaded(self):
+        global interrupted
+        if interrupted or got_sighup:
+            return
+        if self._config["slave-load-max"] is None:
+            return
+        load_max = float(self._config["slave-load-max"])
+        if load_max < 1.0:
+            return
+        if os.getloadavg()[0] <= load_max:
+            return
+        load_resume = max(load_max - 1.0, 0.9)
+        secs = random.randrange(30, 90)
+        while True:
+            load = os.getloadavg()[0]
+            if load <= load_resume:
+                break
+            logging.info("Sleeping due to high load (%.2f)" % load)
+            try:
+                time.sleep(secs)
+            except KeyboardInterrupt:
+                interrupted = True
+            if interrupted or got_sighup:
+                break
+            if secs < 300:
+                secs += random.randrange(30, 90)
+
+

This has the potential to oscillate in the presence of multiple
slaves. I'd recommend reducing the secs range to under the loadavg
time constant (say to 30-60 seconds, 15 to 45 would be better), and
skipping the delay accumulation.

If you are in a situation where 30 seconds isn't long enough, making
it longer is not going to help.

It could also help to add a bit of noise to the load_resume threshold.