[Soc-coordination] Last Report improving the online package review interface for mentors.d.n

Sun Aug 19 19:55:52 UTC 2012

Dear Mailing list,

Here is the last report on improving the online package review
interface for mentors.debian.net (debexpo),

_ Summary for our friendly org admins _

        My summer of code project, "Improving the online package review
interface for mentors.debian.net", intended to add and
    improve some of the features of debexpo. This software, while in
production for a year now on mentors.debian.net,
    needed some work to be more adapted to the workflow of its users.
During this project, I implemented a code browser
    with revisions and online threaded comments, to allow for easy online
reviews. Those comments can be rated by
    other users for "karma", and can be summarized and sent by email to,
for instance, the sponsorship-request bug,
    hence keeping track of progress and accomodating other users'
workflows. Provisions have also been made
    for the integration of specific workflows for packaging teams, although
they need to be completed.

_So what's done yet?_

__comment engine__

        This was the biggest part of my work, I didn't expect to work that
much on it but I got a lot of issues very hard to handle,
    I had to change a lot of things in the database in order to have
something clean,
    yet it's almost clean but If I want to have something perfect,
    I need to change all of debexpo's database model declarations and it
will take too much time.

        My first try on database resulted in painful errors, I got cyclic
imports, not for 1 or 2 loops but for at least 3 loops,
    I found a solution on the Great Internet that looks ok. Documentation
on cyclic imports with python isn't really helpful,
    I learned that it was one of the most painful issue with python, and I
learned that there is no perfect solution,
    and I should try some small tricks.
        After these first issues, I realised that I was not able to use the
same validation decorator with ajax and later with
    standard POST, I found a solution that acts exactly as the decorator
but with human readable code.

        The first working samples were with ajax and worked pretty fine, I
changed a lot of things in the user interface
    during this part, and that's why my first works on message service was
really helpful. Now posting comments is
    working and if it fails, you will have a fancy little message that
tells you the error.

        As posting comments was now working I focused my work on the thread
system, this was not as easy as I thought,
    I wanted to do as reddit[0] does, while using a type of comments (good
things, bad things, information, idea)
    in order to have something that you can browse fast if you are lazy.
Answering to a comment was just about
    adding a field to the form, to refer to the parent comment. this was
quite easy but I made the best list builder
    that I could in order to output threads. Unfortunately, this didn't
work, there were about 5 lines and somewhere in these 5 lines something was
wrong.
    I asked for some help and Nicolas noticed that I wasn't using my
relation in the good direction,
    I was using the fact that a comment had a parent instead of using the
fact that comments could have replies.

        So I worked again on the database and I noticed that I wasn't using
any relation,
    I finally understood that it was because of my cyclic import issue. I
tried it again and again,
    and finally asked again for some help then I realise that I shouldn't
have cycling import if I'm doing things in the right order.
    Finally, the relation 'comment answers to another comment' (child has a
parent) works. With my code
    it should work fine but it may be better to work with a relation
'comment got answer' so I tried again and this time
    I couldn't avoid the cycling import. Nicolas sent me the comments model
using another type of model declaration
    but this kind of declaration (the "declarative base" in SQLAlchemy)
required a change on all model declarations.

       I finaly solved my issue with comment sorting, I thought that python
would be able to compare 2 objects with their
    id but it's not possible (and not useful). so I fixed that and it was
working fine.
    But this isn't the best way to do this and I should change all the
model declarations one day.

        The next step was the rating on comment, this was as easy as adding
a new form on the page excluding some form issues,
    this was simple to solve with pylons' webhelper. It's currently working
fine with or without js.
    As requesteed, the best rated comment and the latest one are listed on
the main page. For the latest it was quite easy
    but the first one used a filter that sorts comments from lower karma to
higher karma, then comments without karma
    and I wasn't able to get the best comments, I had to remove all the
None values before getting the best rated comment.
    I would like to produce a work that can work at every place you want,
so I split it in small part in order to get exactly
    what you want, for example in the index page, I need only something
that display 1 comments, and 1 other.
    this looks really easy with a mako template inclusion, but for each
included stuff I lost almost all the variables from the context.

        I lost time trying to solve this with "global" variables but this
wasn't the good solution,
    I needed to use mako includes with arguments. Of course docs on it
exist but it was pretty hard to find because
    it wasn't used in the same way than I did. This changed a lot of things
in the mako templates but it's working better
    than expected. Also it makes ajax answers safer because instead of
sending the User objet
    (with every things including password *cough*) I'm sending only the
name. (Of course this security was planned
    but the solution was found before I noticed this problem)
    At each time I read again my code, I found a lot of things that could
change in order to get something better and better
    so I will be able to make better version of this soon.

__git storage__

    I was working with a not-obvious link between Package's version and git
storage, as simple as new version == new commit
    than access to the 3rd package's version result in access to the 3rd
commit.
    This was easy to understand ( as soon as you got it, this is a little
paradoxal, I know) but not really suitable
    to the work needed, so I added the "commit id" to the database and
changed the parts of the project that needed it.
    it was quite easy and may help other developers with this tool, docs is
now available.

_what is missing?_

    Activity tracker isn't here, it was started but not finished for the
first deadline. But the work wasn't really good
    as the project is not really 'user' aimed but more 'package' aimed
(which is more logical)
    Syndication interface isn't started, time was missing to start another
chunk of work, even if how I planned it,
    it was supposed to be easy to do.
    Team integration, this is very sad that I didn't get time to finish it,
I know that I started to think about it
    because it was one of the most important things for the project, I know
that I did some work on it in my first commit,
    but I didn't find the time to end it.

_what were the main difficulties?_

     As I said before, I had a lot of difficulties with the python
language, cyclic imports were the most hard to handle,
     the small one with the equal (I may think more in 'object'). I got
difficulties with the form validation system
     that required some searches on the Great Internet.
     Again I spent a lot of time with the database, because of cyclic
import and because of something that wasn't on the docs
     of the project (and is now), the database wasn't setup by the project,
I had to do it by hand,
     and it's because an import was missing in the initialization file.
     Time and organisation was the last hard point of this last part of the
project, I did the best possible but
     I have too many things to do and I have to make good work.

_What is next?_

      Excluding gsoc stuff ( evaluation, code submission), I have to merge
my branch to the current project,
  solve the possible issues, then I would like to do the missing stuff you
could read before.
  Finally, there is room for improvement on the comments engine.

_All this sounds cool, but show me some evidence!_

    comments : http://kolo.fr/comments.html
        this is a sample of the comments engine with threads,
        comments are hidden by default, a panel will apear if you click on
the button
        it won't annoy you if you need to use the horizontal scroll
        it will still work if you turn off javascript

_last words _

        This is the last report and I would like to end it with several
thanks to all the people that help me / cooking-giving / make gsoc
possible,
    especially to my mentors and other people on #debexpo. I learnt a lot
of things during this few months
    and I hope the work I did will help some packages get into Debian!
    Thanks for reading!

Again Feel free to query me on Irc for any question, idea, or information!
Good luck!
ko_lo

[0] http://www.reddit.com/r/debian/comments/bhd6u/thank_you_debian/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/soc-coordination/attachments/20120819/9f299025/attachment-0001.html>