[Soc-coordination] Re: Deciding on our applications

[Steve McIntyre]

[ If none of this makes sense, please read the archive to catch
  up. I've just subscribed the last few mentors whom I couldn't
  identify as already being subscribed. ]

On Tue, Mar 27, 2007 at 03:03:55PM +0100, Steve McIntyre wrote:

[ how to spread the load of reviewing apps ]

>I suggest:
>
> 1. Mentors identify all the projects that match up with their own
>    suggestions (e.g. I'll take all the various CD-tester apps) and
>    identify them here on the list as ready for review
>
> 2. For each application that doesn't match our suggestions, please
>    volunteer to act as a reviewer for it if you know the area
>    well. If there are any more with no primary reviewers, then admins
>    will assign to mentors who look comparatively free.
>
> 3. Once we have a primary reviewer organised for each application, we
>    can work from there. For each application, I suggest we pick five
>    others of our mentors (randomly) to also read the application and
>    provide comments and evaluations. If there any other projects that
>    you have strong feelings on, please dive in also. It is up to the
>    primary reviewer to act as a guide in terms of the application
>    details if necessary (e.g. I know nothing about OVAL, but can help
>    explain the CD tester app)
>
> 4. We total up the scores for each application, and normalise the
>    scores appropriately to give us a reasonable spread at the top.
>    e.g. we don't want 5 of the CD tester applications when other
>    potentially good projects may score just slightly lower.
>
> 5. Last round of comments / reviews. Maybe an IRC meeting of
>    interested mentors (TBD) for any last ordering tweaks
>
> 6. Ready for submissions to Google

Nobody has argued with this or suggested anything better (that I've
seen). AJ has come up with some rankings of how many applications we
have in each area, which should be useful. He has also assigned
suggested mentors to several of what he has guessed to be the
best-looking projects. Please check if you're one of those people, and
if you don't like what you've got then let us know.

Time is getting tight. For the rest of the process above, I propose
the following timescale:

 1 - By Sunday night (1st April), please identify projects you would
     like to mentor/review. If you're happy to spend time to review
     *all* the projects (as several admins and Wookey appear to be),
     then please mention that too. All responses to the list please.

 2 - Monday 2nd: Admins will pick up any projects not so identified
     and will assign primary reviewers for them. We'll then pick
     remaining reviewers for each. The plan is to get *at least* five
     reviews of each app.

 3 - Tuesday 3rd to Thursday 5th: Go reviewing! If you're not sure
     about how to rate things, ask the primary reviewer (for technical
     info) or the admins.

 4 - Friday 6th to Saturday 7th: admins normalise the scores with help
     from primary reviewers and we sort out our final ordering ready
     for Google.

 5 - Wednesday 11th: (in theory) we hear from Google what their
     decisions are. They tend to extend deadlines (as evidenced this
     year and last year), so there may be some delay on this and then
     some discussion between mentors, students and Google to work out
     *exactly* what the final assignments will be.

Go!

-- 
Steve McIntyre, Cambridge, UK.                                steve at einval.com
"Because heaters aren't purple!" -- Catherine Pitt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.alioth.debian.org/pipermail/soc-coordination/attachments/20070330/4b0276b9/attachment.pgp