[Soc-coordination] Report 3 : Create Lintian reports frontend

[Abhishek Kumar]

This is the third report for my project : Create Lintian reports frontend
[0], mentored by Niels Thykier.

I started last week by writing down the proposed approaches about things
that need to be implemented in harness - distributed execution and working
directly using http.
After that I started working on making harness work over http rather than
having the need for a local repository.
The approach in a nutshell is to get the binaries and sources of the
packages that need to be analyzed from an online repository using the http
protocol. Once, the required files are available locally, run lintian over
them. And delete the files after they are analyzed.

The source code for the work done till now is available at [1]. The code
available is able to do all the steps that I have mentioned below except
for the 4th one.

So, now delving into details about how the feature of http access is being
implemented for harness.

Step 1 : Set the various parameters like the url for the distribution,
architecture and area in the reporting/harness.config file.

Step 2 : Run reporting/harness.pl.
    Niels has created a mock apt configuration and apt-get with this custom
configuration is being used for retreiving the files. The configuration
files can be found inside the directory apt-mirror-test inside my
repository. The location of this directory is stored in $APT_ROOT.
    The harness.pl script will first read the harness.config file and based
on the various parameters it will generate custom apt.conf and sources.list
fies which will be stored in $APT_ROOT/etc/apt.

Step 3 : Now harness.pl will execute the command $RUN_APT(details can be
found at $APT_ROOT/README) update which again is a custom command which
will retreive :
    Packages.gz and Sources.gz files available inside the repository and
store them inside $APT_ROOT/lists. Suppose we are using the repository
http://ftp.debian.org/. Then :
    Packages.gz is available at
http://ftp.debian.org/debian/dists/$dist/$area/binary-$arch/
    Sources.gz is availabe at
http://ftp.debian.org/debian/dists/$dist/$area/source/
        $dist refers to the distribution like sid, squeeze etc.
        $area refers to main, contrib or or non-free
        $arch refers to the architecture like i386, amd64 etc.
 Packages.gz file is specific to each architecture and contains the
detailed information about the binary packages available on the repository
while the Sources.gz file is the same for all the architectures and
contains detialed information about the source packages available on the
repository.

Step 4 : After the required files are downloaded, they will be parsed to
extract the information about the binaries and packages. This piece of code
is currently missing from the script. This functionality is already
available in the old harness code. But, currently I am not using it due to
some issues that were there and we are working on writing the code that
will be able to address the past issues.
What is basically missing here is the functionality to read the index files
and create the data structure to hold the essential information.

Step 5 : The binaries and packages will be downloaded inside a temporary
directory($APT_ROOT/downloads) and lintian will be used to analyze the
package. After the analysis is done, files will be removed from the
temporary directory.

--------------------Next Steps--------------------

Write the missing code.
Add the incremental runs feature.

----------------------------------------------------------------------------------------------------

[0] :
http://wiki.debian.org/SummerOfCode2012/StudentApplications/AbhishekKumar
[1] : http://anonscm.debian.org/gitweb/?p=users/abhi-guest/harness.git

----------------------------------------------------------------------------------------------------


Abhishek Kumar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/soc-coordination/attachments/20120702/51493fa8/attachment.html>