[Soc-coordination] Fourth report: Debsources as a Platform

[Clément Schreiner]

Hi,

this is the fourth weekly report on my Summer of Code project 'Debsources
as a Platform'[1].

(Previous report:
http://lists.alioth.debian.org/pipermail/soc-coordination/2015-June/002511.html)

0. how to run hooks
-------------------

This task was mostly done last week, but I added more information which
will also be useful in other areas. Merged in PR #16.[2]

I found solutions for running tasks on the same machine as another,
which would allow running the hooks in a single transaction should we
want that.

1. how to handle the failure of hooks
-------------------------------------
 
Once we decided on a solution for running hooks, that was easier: PR #14[3]

2. update the prototype with sqlalchemy queries
-----------------------------------------------

Those are simple changes: PR #13[4]

3. define which tasks need what data
------------------------------------

In order to allow us running some tasks remotely, I figured out which
tasks needed to access the database, the mirror or the extracted
sources. I updated the previous graph of tasks. Merged in PR #17[5].

4. data locality
----------------

On the same subject, I expanded the documentation work I started in the
"how to run hooks" task for running tasks on specific workers/machine,
also merged in PR #17.


---------
Next week
---------

Next week, I will finish the first version of the protocol, and start
implementing the extract_new stage with support for hooks. This differs
from the initial plan (implementing all the basic stages without
plugins), but since the hook support will be the more complicated task,
I think it is best to start with that, in case some unexpected issues
break the design.

Thanks for reading.

Cheers,

Clément

[1]
[https://wiki.debian.org/SummerOfCode2015/StudentApplications/ClementSchreiner]

[2] https://github.com/Debian/debsources/pull/16/

[3] https://github.com/Debian/debsources/pull/14/

[4] https://github.com/Debian/debsources/pull/13/

[5] https://github.com/Debian/debsources/pull/17/