[Soc-coordination] Fifth report: Debsources as a Platform

[Clément Schreiner]

Hi,


This is the fifth weekly report on my Summer of Code project 'Debsources
as a Platform'[1].

(Previous report:
http://lists.alioth.debian.org/pipermail/soc-coordination/2015-June/002530.html)

This week I worked on the extract_new stage. My work has been light
commit-wise, but I'm on a good track, and my asynchronous
debsources-update is now able to extract the sources of packages and
fill the database with basic information on packages.

I had two main problems:

1. When running the update, only the first package would get extracted
correctly, with the next dpkg-source executions raising an error. It
took me a lot of time to track down to a simple mistake I'd made in the
prototype[2].

2. I had trouble[3] using celery's 'worker_direct'[4] function for
queuing a subtask on the same worker as the current task. The online
documentation was light on the subject and I had not found examples of
how to uses it. I finally managed to make it work[2].


Next week:

1. Port the hello world plugin to a celery task

2. Update the testsuite for the new updater

I will need to update the tests for the library changes I needed, as
well as configuring celery for new running tasks synchronously in unit
tests.

3. Update the actual plugins to celery tasks

Thanks for reading,

Clément

[1] [https://wiki.debian.org/SummerOfCode2015/StudentApplications/ClementSchreiner]

[2] https://github.com/clemux/debsources/commit/4708227a6446c5afc7999d85c155ee10ab4ae328

[3] [http://stackoverflow.com/questions/31073110/celery-how-to-use-celery-utils-worker-direct]

[4] http://docs.celeryproject.org/en/latest/internals/reference/celery.utils.html#celery.utils.worker_direct