[Piuparts-devel] for preview: preview/submit-after-processing (was: Re: submitting logs after section has finished?)

[Andreas Beckmann]

Andreas Beckmann (6):
      p-s: add _count_submittable_logs() method
      p-s: factor out _talk_to_master()
      p-s: talk to master only if we submit or reserve logs
      p-s: submit logs after a section was processed
      p-s: submit logs after (single) Ctrl-C
      p-s: try to unreserve all reserved logs after interrupt

(on top of 4holger)

On 2012-06-20 16:26, Holger Levsen wrote:
> Hi,
> 
> On Mittwoch, 20. Juni 2012, Andreas Beckmann wrote:
>> what do you think about submitting the logs after a section has been
>> finished before moving on to the next? This will reduce blocking other
> yes, that why I *want* this :)

There you go ...

I'm not perfectly sure about the last two patches - try to connect to
the master in Ctrl-C case?

> (though I first want to release 0.45 before I will merge more stuff...)

it's not really an invasive change ...

> I think its also useful for single slaves... 

from a reporting point of view: yes, but not from a slave starving point


This approach is also not useful for slaves serving exactly one section
- will the be a common case for quick retesting of a section? In that
case doubling the number of connections to master may be harmful.


On 2012-06-20 16:22, Dave Steele wrote:
> I'd consider a couple of steps further. Submitting completed work across
> sections should be job #1, above processing sections. And it could be good

That would get much more invasive, but I'll look into it. We need to get
experience it its worth the effort with the current patches in place. So
whenever the master is busy for that section, the slave won't be able to
send the stuff ...

> for slaves to submit logs in batches much smaller than max_slaves as they
> process a given section (given that the process is cheap). (this would be
> an alternative to master caching work available, which would drive
> max_slaves itself down).

If the slave is buffering a sensible amount of packages, e.g. 1 hour,
would we really need this?


Andreas