[Debian-l10n-devel] Reducing the time gap to update the stats pages

[Felipe Augusto van de Wiel (faw)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 02-06-2008 19:25, Nicolas François wrote:
> On Mon, Jun 02, 2008 at 06:41:41AM +0200, Christian Perrier wrote:
>> As of now, it seemms to me that pages like
>> http://www.debian.org/intl/l10n/po-debconf/fr
>> are updated quite late after packages are installed in the archive.
>>
>> For instance, the French stats show mailagent as incomplete while the
>> updated package was uploaded on May 31st by Manoj....
>>
>> I don't exactly remember how these stats are generated. It involves
>> churro, IIRC, but not only.
> 
> I don't know neither how the web stats are generated.

	lessoften from cron in the Debian CVS:

		http://cvs.debian.org/cron/lessoften?rev=1.24&root=webwml&view=markup


	It runs once a day:

		http://people.debian.org/~faw/tools/wwwbuild-lessoften.html



> I think they are using the unstable material (the unstable.gz in the data/
> directory).

	Yes, they grab it from churro, and inside lessoften it uses
scripts from international/l10n:

http://cvs.debian.org/webwml/english/international/l10n/scripts/?root=webwml



> To reduce the delay, the only right way I see would be to generate the
> stats from churro and push them to the we site, or just to have a link
> from the web site to churro.
> 
> I could also send a trigger to a robot to the web site host, but I find
> this too complex.

	You can't do that. The website has to cycles, one runs more
than other and both have impact on the webmirror infrastructure, and
DDs also don't have access to the webroot, you need to be in the
webmaster group to direct sync things in there, but that would broken
the translation structure.

	Basically, we can work to improve things, specially merging
Helge change to integrate metadata and ask them to be in 'often'
instead of 'lessoften', we can parse it on churro to relief the
load during the build and produce pre-parsed material.


> On churro, some cron job are scheduled after the others because we know
> there are some dependencies. It might be better to have a superset script
> which deals with the scheduling. This might avoid some issues if a job
> takes more time than expected, and this might reduce the delay needed to
> receive the result of the jobs (I usually don't care since I'm sleeping at
> that time)

	That's indeed a good idea.


>> And, anyway, the way these pages are updated *should* be documented
>> somewhere. For me, this is kinda black magic and being the de facto
>> i18n guru, I don't like that..:-)
> 
> I agree, and I added recently links to this mailing list to the web pages
> generated on churro, and I try to make sure the scripts used to generate
> stats are available publicly.

	I think we should not host anything under our $HOMEs anymore,
that includes move Eddy and other stuff under robots and properly
use group permission, because we hardly will have root once we move
to DSA.


>> Nicolas, that's probably for you, again...
> 
> I will try to find a web guy to check how it works (not before this
> week-end).

	Hey, I'm sorry I'm disappeared, that's the end of the semester
and I'm doing extra courses because I want to graduate by the end of
the year, I'm not dead. :-)   Vacation should start next week and it
allow me to work with Nicolas on the pending reports, TODO list, i18n
Extremadura meeting and all other stuff, but for now, let's work on
the urgent things. (Sorry again).

Kind regards,
- --
Felipe Augusto van de Wiel (faw)
"Debian. Freedom to code. Code to freedom!"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIV9ubCjAO0JDlykYRAjQXAKCr3lQQIzqzFrVPBhurO6F8yThPqQCgwp5e
+bf9Wd/gGgb5Vsl8s6/XpdA=
=Cbga
-----END PGP SIGNATURE-----