[Teammetrics-discuss] Help needed to parse alioth lists

[Andreas Tille]

Hi,

I have tried to port the teammetrics code from Python2 to Python3.
Unfortunately I'm facing some encoding problems in only a few mailing
lists.  You can see the broken text data (and the resulting graphs
which show the issues of the names directly) here:

    http://blends.debian.net/liststats/_tmp_/broken_names/

I stripped down the teammetrics code to those parts of code that
are parsing the lists at alioth-lists.debian.net and moved it to
the branch

    https://salsa.debian.org/teammetrics-team/teammetrics/-/tree/debug_read_alioth_lists

(Shayan, I made you team member so you have push permissions.)

In

    https://salsa.debian.org/teammetrics-team/teammetrics/-/blob/debug_read_alioth_lists/etc/teammetrics/listinfo.conf

I have set only one single list which shows the issue - no need
to download more list archives for the moment.  You can simply
run the script

    https://salsa.debian.org/teammetrics-team/teammetrics/-/blob/debug_read_alioth_lists/liststat.py

but before running this you need to create the dirs

   /var/cache/teammetrics
   /var/log/teammetrics

and grant the user who runs liststat.py writing permissions to these
dirs.  In /var/log/teammetrics you will find all log lines created by
logger in debug level.

I think there is no need to create the database where those data will be
commited after everything is read and converted - so some more debug
code in advance should do to find out.  In case you want to have the
database anyway may be the easiest way to create it is to import a
backup.  Here you can find the latest backup and a script for importing
it into an existing psql server:

   http://blends.debian.net/liststats/_tmp_/database_backup/

Feel free to ask back in case you need some more information.

Kind regards

      Andreas.

-- 
http://fam-tille.de