[Piuparts-devel] for preview: preview/submit-after-processing (was: Re: submitting logs after section has finished?)

[Dave Steele]

On Thu, Jun 21, 2012 at 7:27 AM, Dave Steele <dsteele at gmail.com> wrote:

>
> I'm not sure. Try 1) adding a 'push only' option to the slave connect
> (needed for both of our strategies),  2) getting a list of sections with
> finished work at the top of the loop, sorted by priority, and 3) if the
> list has more than one entry, doing a 'push only' for all but the first.
> (2) should be the most invasive part, and it doesn't look bad.
>
> ...
>
> ... There is a small race. Change (3) to flush all sections with work, to
> fix the race. This restores the need for the second connect in the normal
> case. ... or maybe look at the push after a section has been successfully
> worked (i.e. at the end of the loop). This needs some thought. It is worth
> trying to avoid the second connect in the typical case - the invasive
> option here is getting master to not parse 'Packages' in the 'second
> connect push' case.
>

For posterity:

- Run the section normally, with the usual push at the beginning of the
master connection.
- At the end of the loop, collect the sections which have push-able work
- For every section except the section that did work on this pass, connect
and push the work

Under typical conditions, uploads happen immediately, and the master is
connected to only once per section per loop. In other cases, all work is
(attempted to be) pushed on the next iteration (instead of 'eventually),
regardless of busy conditions.

I'd have to look again, but it seems very close to what you have.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/piuparts-devel/attachments/20120621/6dcbe2fa/attachment.html>