[Piuparts-devel] faster-rdep-calc

[Dave Steele]

On Sat, Mar 9, 2013 at 8:01 AM, Andreas Beckmann <anbe at debian.org> wrote:
>
> Do you expect a speed improvement if the rdep count is needed for *all*
> packages (as done by p-report)?.

I don't expect a speedup of a sort/inspection of reverse dependency
information for a full list of packages, unless the new set-based
package aggregation loop is faster than the old recursion function. I
wouldn't be too surprised if it is slightly faster, but I have no
data.

piuparts-report doesn't access rdep info for all packages. If you have
one of my later commits (and I think you do), the rdep access/sort is
avoided for successfully-tested - hence there should be a time savings
for each section that is on the order of what master will see. I
thought I saw a bit less than 10 seconds, with idle slaves.

 I'd expect an improvement for p-master
> if you only need the values for waiting-to-be-tested (and their rdeps),
> and that usually amounts to much less than all packages.
>

I do the (possibly) expensive recursive calculation only once per
package, and use the same results for rdep count, block count, and
waiting count. What you describe implies doing this separately for at
least some cases. This might be the right thing to do for master, but
I think the optimization may be in the wrong place. It helps most when
the package list is small, which is the case that doesn't need the
extra help this would provide.

> I also merged rdep-chain-len for local testing :-)
> Do you use "full" depth or only "blocked" depth?
>

Full depth, as defined by Package.dependencies(), meaning that
'alternative' dependencies are not being considered.

The point of chain-len is to drive the waiting-for-dependency list to
zero faster for a full retest. It didn't make a significant change in
this case, though I still think it should be considered as a help for
smaller work loads.

The next step here is to add alternatives to the reverse dependency
processing. That should take care of that issue. This will have ripple
effects across the metrics. The new numbers reported may be considered
to be better or worse.