[Soc-coordination] Last report: Provide some metrics for Debile

[Clément Schreiner]

Hi, this is the thirteenth and last weekly report on my Summer of Code project 'Provide
some metrics in Debile'[1].

(Previous report:
http://lists.alioth.debian.org/pipermail/soc-coordination/2014-August/002274.html)


broken sbuild
-------------

Sadly, all Architecture: all packages still can't be built on debile
because of the incompatibility between dpkg 1.17.11+ and sbuild.

I wrote a small patch that was supposed to fix that[2]. However, while
neither sbuild nor dpkg-buildpackage complain, no .deb or .changes file
is generated. I've been stuck on that for most of the week, and can't
figure out what goes wrong. I asked Léo Cavaillé (2013 GSoC
student, who has experience with sbuild's codebase) for help, and he started
to look into the problem.


coccinelle
----------

Meanwhile, I made sure coccinelle[3] would run fine when we can finally
launch the long-promised rebuild. To deploy the coccinelle patches on
the debile-slave blades, I used a nice feature of Salt,
salt.states.git.latest[4], which allows specifying a git repository to be
cloned and kept up-to-date on all salt minions.

uniquify
--------

Once coccinelle was functional on ursae, I ran into a bug, and then another, in my new
implementation of uniquify[5]. Some types of objects would not be
inserted correctly into the database[6], and others were not inserted
in the right order[7][8] (for some reason, my statement "set constraints
all deferred;" does not work, which means tables must be filled in the
right order to avoid integrity errors).

ricky
-----

When issuing a rebuild for a set of packages, ricky was sometimes
crashing with this traceback:
   File "/usr/lib/python2.7/dist-packages/debian/deb822.py", line 251, in __getitem__
    value = self.__dict[key]
  KeyError: 'Checksums-Sha1'

An example of package that fails:
 http://ftp.debian.org/debian/pool/main/x/xicc/xicc_0.2-3.dsc

After some research, I learnt that the Checksums-Sha1 and
Checksums-Sha256 fields are mandatory only since Debian Policy version
3.9.5.0, released in October, 2013, though they've been recommended for
4 years (3.9.0.0)[9].

Some packages using very old standards versions (xicc, for example,
triggers the lintian warning 'ancient-standards-version 3.7.3 (current
is 3.9.5)') thus can't be uploaded using ricky at the moment.

I'm not sure how ricky should handle that sort of packages, since there
might be other incompatibilites, but in the meantime I added a check for
missing fields in the DSC[10], so that it does not crash anymore, and
continue uploading packages.


end of the GSoC
---------------

That's it, the summer of code is over. However, in addition to fixing
sbuild with Leo and launching the rebuild (hopefuly before or during
Debconf), I want to continue working on debile:

 - while developing metrics for debile-web was not possible because of
   the state debile was at the beginning of the gsoc, I'm still
   interested in doing that, and improving the web UI

 - some checks that were working before have not functional for almost a
   year, and I'd like to help fix them

 - and various other improvements (new checks, …) that I've thought of doing during the
   summer and did not have the time for


Thanks for reading, 

------
Clément

[1] 
[https://wiki.debian.org/SummerOfCode2014/StudentApplications/ClementSchreiner]
[2]
http://www.mux.me/debile/0001-Add-only-arch-all-for-passing-A-to-dpkg-buildpackage.patch
[3] http://coccinelle.lip6.fr/ 
[4] http://docs.saltstack.com/en/latest/ref/states/all/salt.states.git.html
[5]
https://github.com/clemux/firewoes/blob/iterative_uniquify/firewoes/lib/uniquify.py
[6] https://github.com/clemux/firewoes/commit/6861fba6e12f7f5184299088dcbce7a6d21a3e33
[7]
https://github.com/clemux/firewoes/commit/7fcab8ed76f60dba6880b0033c91db4ed4523045
[8] https://github.com/clemux/firewoes/commit/408f142f0136dcec17ae70a45c1dc9328beaa3d7
[9] https://www.debian.org/doc/debian-policy/upgrading-checklist
[10] https://github.com/paultag/ricky/commit/69ec973acc0ec4ed2e5edd44aba0a503aa06332b