[Popcon-developers] is there a historical data per participant?

[Bill Allombert]

On Wed, May 09, 2012 at 10:42:32PM -0400, Yaroslav Halchenko wrote:
> Hi guys,
> 
> Thanks for taking care about the popcon -- it is a unique and
> irreplaceable resource, and so far it seems to work ok even on a
> 'server' side for our http://neuro.debian.net/popcon where we
> complement users' /etc/popularity-contest.conf with
> SUBMITURLS="$SUBMITURLS
> http://neuro.debian.net/cgi-bin/popcon-submit.cgi" thus enabling them to
> submit to both Debian (and Ubuntu) and us (NeuroDebian).
> 
> From time to time we are trying to get a better sense of dynamics not
> only of how many users of any particular package there are, but how many
> users of a particular kind (i.e. using some number of packages from a
> given set, e.g. packages we maintain in Debian).  Clearly that is
> impossible to deduce based on available summary files (located under
> /srv/popcon.debian.org/popcon-mail/all-popcon-results).  It would
> require historical data for
> /srv/popcon.debian.org/popcon-mail/popcon-entries, i.e. submissions per
> each participant in the past.
> 
> From a brief look I do not see such data available anywhere on
> popcon.debian.org (i.e. popov.debian.org) but I cherish the hope that
> may be someone was collecting such data (e.g. properly archiving it into
> some DB; or via some other means, e.g. as wild as tracking the content
> of popcon-entries under GIT or some other VCS).  Is there such
> historical data available anywhere and accessible (to DDs)?
> 
> May be the content of this host has previous backups stored
> somewhere so at least scarce snapshots of past points could be
> recovered?
> 
> If not, would you consider it feasible to setup such historical data
> capture at least going forward from now on?  I would be glad to
> contribute the implementation and just wanted to check first on
> the necessity and feasibility ;)

Hello Yaroslav,

We do not store historical data of users submission for privacy and security
reasons:

The anonymization is rather weak.  Matching subscribers to reports can be
achieved through various means and if the data were leaked, the whole history
of packages of each users would be leaked.

If you are doing that yourself, I think you should warn users of the implications.

Cheers,
-- 
Bill. <ballombe at debian.org>

Imagine a large red swirl here.