[Soc-coordination] Second report: Debsources as a Platform

[Clément Schreiner]

Hi,


this is the second weekly report on my Summer of Code project 'Debsources
as a Platform'[1].

Previous report:
http://lists.alioth.debian.org/pipermail/soc-coordination/2015-May/002463.html

This week I continued working on designing the asynchronous architecture
the updater.

How to send SourcePackage to tasks
----------------------------------

We decided to create a python dictionary from the essential data in
SourcePackage and send that to the tasks, and I implemented that[2].

Prototype
---------

I then updated the skeleton of the new updater[3]; it will allow me to
experiment things when designing the architecture more thoroughly.

Graph of tasks
--------------

As the first step in designing the architecture of the new updater, I
made a graph of all the tasks[4][5] using graphviz.

------------
Next week[6]
------------

I haven't been able to finish the design this week, as I initially
planned, but the graph of tasks will help me. It will soon be time to
start working on the implementation, and the project should progress
faster then.

1. Problem: how should we handle the failure of a hook execution

In the current, synchronous implementation of the updater, the failure
of a hook (plugin) will trigger a session rollback, and the addition (or
removal) of the package from the DB will be started from the beginning
again. We can't do that in a distributed architecture (as far as I
know), and I need to figure out how it should be done.

2. write the specifications for the updater

3. implement the extract_new stage (without hooks)

4. implement the update_suites stage

5. implement the garbage_collector stage

6. implement the update_metadata stage




Thanks for reading,

Clément



[1]
[https://wiki.debian.org/SummerOfCode2015/StudentApplications/ClementSchreiner]

[2]
[https://github.com/clemux/debsources/commit/80903186ef65c39f07a6a1bff742076040f3817f]

[3]
[https://github.com/clemux/debsources/commit/beb2b23103ac013627185cbfc3cbd5362c5e0359]

[4] [https://github.com/Debian/debsources/pull/9]

[5] [http://www.mux.me/debsources/async_tasks.png]

[6] [https://trello.com/b/LG8eUfPS/debsources]