[Piuparts-devel] Thoughts on masterd

[Holger Levsen]

Hi Dave,

On Samstag, 3. Dezember 2011, Dave Steele wrote:
> On Sun, Nov 27, 2011 at 4:25 PM, Dave Steele <dsteele at gmail.com> wrote:
> > ...couple of problems with piuparts-master:
> > I've been thinking ...
> > ...idea:
> > ...humbly propose...
> > Thoughts? ...
> 
> It seems that I don't have a clear sense of what the consensus goal is
> on this topic, I'll concentrate elsewhere.

of course you're free to come to this conclusion, but let me assure you that 
I'm all for making piuparts-master faster. And maybe your proposal is the way 
to go! I just don't know... 

The problem is, your plans are quite intrusive and we have a reliably working 
master atm. So if you want me to replace that code with something else, you 
need to show me the code, not just a mere proposal.

As I see it, improving startup speed of master would gain/free 30-60 cpu 
minutes on piatti per day, at the maximum. While the bottleneck really is disk 
I/O, so this would gain us little, while it introduces instability & work. 
(And you might have noticed that sadly my time for piuparts atm is rather 
limited too.)

You might want to check 
http://munin.debian.org/debian.org/piatti.debian.org/index.html

All this said, having a masterd running all the time sounds like a good idea 
(as long as its not eating too much memory, as memory is used to cache disk-
data thus saving I/O.)

I'm unsure whether to run a second slave on piatti (it has a dualcore cpu), 
but then the real/worse bottleneck is processing the generated logs: there are 
almost 500 detected but unreported bugs in sid and 100 in wheezy! :(  (*)
So sure, we could do better at generating them faster and I'm happy to include 
such patches, but I want do concentrate on filing bugs (and detecting new 
types of issues).

(And piuparts-report should not take a quarter of the day / must run faster so 
we can add more sections + suites. But we just cut that down from 6h to 4h, so 
I'm inclined to add squeeze2wheezy tests soon.)

But as said: give me patches which are proven to work, and I'll be thankful 
and gladly merge them! :-)


cheers,
	Holger

(*) sure, some people directly look at the piuparts results. some even run 
piuparts. but to get most bugs fixed, bugs need to be filed first.