[Soc-coordination] Report next-to-last : improving the online package review interface for mentors.d.n

[Baptiste Mouterde]

Dear Mailing list,

Here is the next to last report on improving the online package review
interface for mentors.debian.net (debexpo),

_So what's done yet?_

__git Storage__

        The first work was supposed to be test with real data from package,
I made something really easy to use, a function
    change() that will handle the import, than I will use the repository
from the git at every place (and help function
    for harder stuff)

        I share this work with dulwich people and I change a lot of thing
after this, I really miss some information in the
    doc and they explain me why I was doing some stuff wrong, and how I can
fix it.

        I add some trouble with os.path.join or split, but I finally found
why it's wasn't working and use string.join/split
    when os.path can't do the job.

        Than I fix it and the importer work pretty fine except some dual
import that I fixed later, I also add log message
    to know what's going on during the import.

        I test this lib with real source, and I had trouble to build them
(missing lib...) but Clément gave me the good
    commands that I was looking for since 2 or 3 weeks and everything went
fine include importer. It was working; I can
    get sources from the repository. While doing the import I fix a lot of
import in the importer.py.
    But first git access show that tree were empty, and mad everything by
hand (building blob, than tree, link both than
    commit) instead of using stage function. But the problem came from two
things, first of all I used absolute path,
    instead of repository path, and a function was supposed to list file
and do.. Nothing.
        Now it's working and tested with several random useful package and
history tracker work fine.

__Version browser__

        I made a first new version with changing url map in order to get
the last revision but I add a lot of small bug each
    time I try to access revision, that's why I decided to rewrite almost
the half of the browser in order to start from
    revision and then handle last version instead of last revision and
patch for revision.

        I made something really cleaner than excepted with logical
function, get and build, that's work together. this take a
    lot of time because I wanted to change all the logic of the controller.

        The new version work with revision better than possible. I added
links to get older and newer revision in order to
    browse it. I fix some commit_id stuff, one of the main goal  was to
have a readable url, and having a commit id that
    take 20 char in you url isn't really.. Readable.

        Than I started the work on diff viewer, I didn't want to work as
github[0] for diff viewer because I think this is
    not really user-friendly, we don't have the whole document, the diff
break it. I first let the difflib build the
    table, it's showing something quite pretty but they were no color, line
numbers and no syntax highlighting. I decided
    to drop this Idea and try with sequence matcher and adding by hand tags
to html.
    This was not really user-friendly because of deleted/inserted lines the
document wasn't really side-by-side.

        I tried again with make_file from diff lib and I discover that it
was better than I thought first, I notice class
    name for change highlighting, and I apply this change to the make_table
function (where make_file build the whole
    file, with legend, anchor html tag etc.. make_table build  the table
only). This was one of the best versions of the
    diff viewer but it was without syntax highlighting.

        I try to do something like, highlight first than put in table or
the opposite but both produce html as text and not
    as html.. I gave up for a mix between both solutions.
        I came back to the sequence matcher, I didn't really use it as it's
supposed to be used, I first imagine something
    that build a list like [ (nb_of_line,type_of _change) ...] and do in
mako a loop over original content and diff content
    at the same time, which check at each line if the line got a style.
This was a too low-level solution. After
    discussing about it with my mentor, I decided to do the list, and apply
the list to the content before displaying it.
    Finally I find a function on the difflib that give me lines that
changes, blank lines, new lines and deleted lines,
    it was perfect and work pretty fine, some choice in this lib are not
the best one for this work but it's working and it's useful.

        As requested I build a diff standard output in order to display
both (side-by-side) and highlighted diff output,
    this was quite easy, excluding encoding issues.

        I also add something that display diff between folder, it use the
dulwich include write_tree_diff that show what
    change between two tree.

        Now I have something working, I have trouble to using the project
repository to show issues that I have, I rather like
    my own work to show my own work :)

__msg_service__
    Ajax is working pretty fine, and a sample show how to use it in the
easiest way, also if we want to use it without ajax
    we will need a global change on mako system in order to add block to
base.mako

__Comments__
        I start the work on comments as I was supposed to, the first work
was on the database, I have to create something
    which links users and comments to have a 'activity' to follow later.
This was quite easy to code but less to make it
    work, I created a new table in my database in order to feet with my own
requirement.

        I begin to work on the controllers to test if the database is
working and how I will handle it, it's work with two
    part, the comments.mako which is used to list the comments and interact
with existing comments and the form.mako
    which is used to create new comments.

        Both part are linked with javascript, but after a small fix, I will
add a no-js version. Also they are made to work
    with all part of debexpo which deals with package.

        The current work is on database, I have to changes some names on
the whole project in order to have something that
    fits the rest of the project. And after that the rating will be ready

_what is missing?_

  Comments tab on diff viewer (few minutes)
  Index (less than 1 hours)
  Stamina (few hours)


_what were the main difficulties?_
       I couldn't end this report without talking about encoding, In all
the test I could make on debexpo, I had
    encoding/decoding issues, always everywhere, I first spend a lot of
time in docs in order to know how to handle it
    write, but you can find a lot of doc about Unicode>string but as soon
as you want the opposite, doc suddenly
    disappear! they was only the find-encodage-by-using-lib solution that
seem working. But for a stupid reason
    (wget say 403 when I tried to download the lib) I gave up for this
solution.
        I finally fix my first problem using a xhtml encoding, and this was
working pretty fine. it was too good for work
    everywhere, as soon as I began the diff viewer it stops working. The
best solution I found was to avoid theses errors
    by using clever trick.
    But with all the work I produce on encoding, I'm able to handle all
type of char (including jap char and ü)

        The second main difficulties was my older working process on the
code browser, I made something working, than add
    other work as patch on the first work. But each time I add something, I
got bugs on the first work, it took at lot
    of time to fix again and again issues in original work (including a lot
of encoding issues). I solve this point by
    rebuilding the whole work with something that could handle all case.

        Having a working database was the latest difficulties of the latest
weeks; I gave up with creating a new database
    with everything new. I add only the table but first I have no id for
comments, then I had the same problem than usual
    'no right to do that'. then it's issues with column, then with data
type and nullable rows..
    I solve most of them and I didn’t find new bugs in latest tests.
    I had also trouble with form validating that doesn't appear to work
properly with ajax, I find a small trick that avoid the decorator


_What is next?_

  I will focus on started jobs, end the comments engine will take few days
(to have everything’s fine). Then the activity
  trackers, then list filter then team integration then syndication
interface.
  at this point there is a lot of work to do but I will have more times
than weeks before, I will try to do as much as
  possible before the deadline, it may be hard to end perfectly all the
stuff that I planed but not impossible.

_this is cool, show me evidences!_

    of course!
    you can find sample on my work environment (I can't do static content
for this it will take too much time) :

    code browser : http://kolo.fr/codebrowser.html
    side-to-side :
http://kolo.fr:5000/package/cowsay/diff/e6..e4/debian/copyright
    diff output :
http://kolo.fr:5000/package/cowsay/difffile/e6..e4/debian/copyright
    folder diff viewer :
http://kolo.fr:5000/package/cowsay/diff/e6..e4/debian/
    revision browser : http://kolo.fr:5000/package/cowsay/browser/e6/debian/
    comments : http://kolo.fr:5000/package/cowsay/browser/d9/README

Thanks for reading!
Again Feel free to query me on Irc for any question, idea, or information!
Good luck!
ko_lo

[0]
https://github.com/rails/rails/commit/7d54fba4218115b77c6775ef9c2257d56b2df577
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/soc-coordination/attachments/20120730/ef161c68/attachment.html>