[Soc-coordination] debmetrics week 8 status update

[Joseph Bisch]

Hi,

This week:

This week I stopped tracking index.html and made manifest2index.py to
generate the index.html file. Now when you run make it generates
index.html. I added a description to the manifest files that currently
gets displayed next to the manifest link on the index page of the
static interface. I made a number of additions to the JS files
imported by the index page, ultimately making it inherit from
layout.html like the rest of the pages so that it would have all the
same files imported. I kept the minified_grabber up-to-date so that it
would download the new JS files. I modified runner_helper.py to sort
by the ts column in the db_fetch() function. If the data is not
timeseries data (like releases) it sorts by the id column. I switched
to a css tabbed interface for the static web interface, instead of the
JS tabs I was using.

Now the push graphs are only updated when data is pushed (and it only
updates the specific graph that had data pushed). I was therefore able
to remove push_runner.py from the crontab. I moved all non-python
files out of debmetrics/debmetrics to the root. I also moved app.py,
push_runner.py, and pull_runner.py to the root. Now when I do my
imports I can just do "from debmetrics.base" instead of needing to try
"from base" too.

On the dynamic interface there was a container that was being
collapsed, because the contents were being floated. This meant the
container had no height to it so it didn't look like it was containing
its contents. I fixed that with some css called a clearfix. I was able
to add a custom parser to the tablesorter library to sort the ts
column. I fixed the path to the model dir in the makefile, since it
was incorrect now that the makefile is in the root. I moved the target
all to the top of the makefile. I did not realize that make runs the
top target by default. I thought it ran all by default.

Now about the dynamic web interface:

I started on the dynamic interface. If you go to metrics.d.n with JS
enabled you can see it. The JS replaces the links normally displayed
on the index page with a tabbed interface. The tabbed interface lets
you switch between the graph and the table. You select a metric from
the dropdown and click add to graph to add the metric to the graph.
Right now when you click the button, it replaces the previous metric.
In the future I plan that it will just add the metric, leaving the
previous metric. The legend lists all the columns for the metric. It
also displays the name of the metric at the top of the legend. That is
a result of a plugin for flot that I am writing. It is located at
debmetrics/static/js/jquery.flot.superlegends.js. The plugin allows
you to toggle the visibility of all the columns by clicking on the
metric name. In the future I plan to make it show/hide all the columns
instead of toggling. I copied some code to show/hide the columns from
the hiddengraphs plugin. I tried just calling the hiddengraphs code
directly from my plugin, but it doesn't seem possible.

Hiddengraphs is used to show/hide the individual columns by clicking
on the label or the colored square. The latest version of hiddengraphs
doesn't support legends that are placed in a container outside of the
graph area, so I reported that issue and used an older version of
hiddengraphs that I have.

The table currently just displays a single metric in an HTML table.
Just like the graph there is a dropdown with a button to choose and
display a metric. The dropdown is populated with all the metrics,
whereas the dropdown for the graph is just populated with the
timeseries metrics.

There are some tasks which I did not complete this week:

I did not fix the errors with the releases and releases_count metrics.
I only gave it a little time, and there were no immediately obvious
errors. The pull scripts themselves are fine.

We did not meet on irc to fix the tests. I guess I was probably
supposed to ping you, but I am available today if you want to ping me.
I did try uninstalling beautifulsoup, but I did not get any error
about the lack of beautifulsoup when I ran the tests.

I did not change all the internal operations to be on unicode like
Piotr recommended. I think I will wait until I switch the code over to
python3, since it is easier to switch to unicode once instead of doing
it for python2 and then having to change everything for python3. I
will wait until python3-flask is in stable to switch to python3. I
know I can setup a testing chroot, but I have enough tasks as it is.

For next week, in addition to the tasks I did not complete this week:

Next week I want to continue to improve the dynamic interface. I will
work on the superlegends plugin some more. I will make the columns in
the legend indented, so it is more obvious that they are a subcategory
of the metric. I will make clicking on the metric name show/hide the
columns instead of toggling them. I will allow you to add multiple
metrics to the graph.

I will work on the table. I will figure out a way to add multiple
metrics to a single table to compare them. Then I will implement it.
Maybe there will be a way to drag columns around and to delete
columns.

I will add a custom graph for the releases_count metric, since it is
too complex for a default graph. I also need to figure out a way to
display the custom graphs on the dynamic interface.

I will look into the wml templates, so that I can make debmetrics fit
in with the rest of the debian websites.

I will also add some more parsers besides csv.

Schedule:

I am a little ahead of schedule. I have push metric support done,
which I am supposed to be working on until the 14th. I think I have
the static web interface done (at least for GSoC). And I am starting
the dynamic web interface, which I'm not supposed to start until the
21st. I think I will have plenty of time for packaging debmetrics.

Joseph