[Soc-coordination] debmetrics week 7 status update

[Joseph Bisch]

Hi,

This week I pulled out common runner code into runner_helper and graph
code into graph_helper. I added a push runner and a route to the web
interface for adding new push metric data. I modified pull_runner so
that it only runs for pull metrics. I updated the tests due to the
previous changes. I fixed the paths so that they are generated as
absolute paths so that the code works no matter what directory it runs
from. I added push_runner to the crontab file so that it runs to
generate the graphs for push metrics.

I used autofmt_xdate() when graphing so that the xaxis labels are
slanted so that they are readable. I fixed some of the manifest files
where pull metrics were accidentally set to be push metrics. I removed
the day locator from the xaxis of the graphs, because with the day
locator there was a label for each day, which made the xaxis
unreadable. Based on Piotr's suggestion, I used with open() in
runner_helper since it automatically closes the file. I modified the
makefile to use a variable for the manifest directory, because in some
places it had a hard coded directory. I encoded the data that is
POSTed to /push as ascii, because when I did not, the data was unicode
which caused problems when I try to search for a matching key in the
models dict. Finally I renamed the rc_bug metric to rc_bug_count to
match the name of the table.

In addition to those changes to the repo, I also did some work on push
metrics on the server. I added a push script for rc_bugs_count[0] and
added it to the crontab. It works and adds data every six hours, since
that is how often the webpage being scraped is updated.

For next week:

I knew there had to be something I was forgetting. I forgot to make a
list of what I will be working on next week.

First of all, something is wrong with the releases and releases_count
metrics. Releases is only displaying buzz with None as the releasedate
and releaseversion. Releases_count was last updated June 10, so
something is causing releases_count to rollback. So I will figure out
what is wrong with those two metrics and correct it.

Second, I want to use CSS for the tabs instead of JQuery UI, which is
what is used now. Since the static interface that exists now is only
going to be used by people with JS disabled, it makes sense to use
some non-JS method of rendering tabs.

Third, I want to have some way of automatically generating the index
page of the web interface, so that when a new manifest is added a link
to the metrics' page is automatically added. I was thinking of
creating a manifest2index.py like manifest2orm. It could be run by the
makefile, but it would take in all of the manifest files as parameters
instead of being run on each individual manifest file.

At that point, I think I am mostly done with the static web interface.
So if I still have time, I will start on the dynamic interface. My
idea is to have some JS that will replace the links on the index page
with a dynamic graph. There will be a dropdown with a list of metrics
and an "add metric" button. When a metric is selected and the button
clicked, it will add the metric to the graph. There would be a list of
metrics already added and red Xs to remove the metric from the graph.

Also, my mentors and I will be debugging the test suite over IRC next week.

I will make the graphs for push metrics only regenerate when the
appropriate metric has data pushed.

[0] - http://paste.debian.net/108141/

Joseph