From jeff at linuxwest.com Wed Mar 2 22:14:11 2011 From: jeff at linuxwest.com (jeff at linuxwest.com) Date: Wed, 02 Mar 2011 15:14:11 -0700 Subject: [Popcon-developers] =?utf-8?q?status_of_executables=3F?= Message-ID: <20110302151411.f50467801c0b453a6c7e53a62af6edb5.9079eadc6c.wbe@email03.secureserver.net> An HTML attachment was scrubbed... URL: From Bill.Allombert at math.u-bordeaux1.fr Wed Mar 2 23:12:51 2011 From: Bill.Allombert at math.u-bordeaux1.fr (Bill Allombert) Date: Thu, 3 Mar 2011 00:12:51 +0100 Subject: [Popcon-developers] status of executables? In-Reply-To: <20110302151411.f50467801c0b453a6c7e53a62af6edb5.9079eadc6c.wbe@email03.secureserver.net> References: <20110302151411.f50467801c0b453a6c7e53a62af6edb5.9079eadc6c.wbe@email03.secureserver.net> Message-ID: <20110302231251.GD27537@yellowpig> On Wed, Mar 02, 2011 at 03:14:11PM -0700, jeff at linuxwest.com wrote: >

Greetings nobile Popcon group;

How does Popcon know which binary executables have been used? Is /usr/bin/stat (or equiviliant) the method Popcon uses to get access times?

Running a simple test executable on my system does not cause the executable's status to change (I am not using noatime either). Does it work for any of you? Or, do you use some other method which is effective? Suppose a I have a package which consists of one executable and I use it frequently, will Popcon notice it?

Most sincerely,

Jeff Wicks

Greetings nobile Popcon group;

How does Popcon know which binary executables have been used? Is /usr/bin/stat (or equiviliant) the method Popcon uses to get access times?

Most sincerely,

Jeff Wicks

Hello Jeff, popcon use the atime. Unfortunately with recent kernels, this only work 100% correctly if the mount option strictatime is used. Otherwise the atime is not 100% accurate (I think the kernel default to relatime) Cheers, -- Bill. Imagine a large red swirl here. From Bill.Allombert at math.u-bordeaux1.fr Thu Mar 3 21:24:01 2011 From: Bill.Allombert at math.u-bordeaux1.fr (Bill Allombert) Date: Thu, 3 Mar 2011 22:24:01 +0100 Subject: [Popcon-developers] status of executables? In-Reply-To: <20110302165403.f50467801c0b453a6c7e53a62af6edb5.6d56b268f6.wbe@email03.secureserver.net> References: <20110302165403.f50467801c0b453a6c7e53a62af6edb5.6d56b268f6.wbe@email03.secureserver.net> Message-ID: <20110303212401.GH27537@yellowpig> On Wed, Mar 02, 2011 at 04:54:03PM -0700, jeff at linuxwest.com wrote: > Thank you for replying so quick. > > Indeed, mounting with strictatime did yield the desired result. I can > see how some "false positives" *could* happen: > > 1) Opening a directory with a GUI file manager changes atime (to obtain > file types). > 2) Executing `file` on a file. Same as above. > 3) Doing a `grep foo *` within a directory opens the file. > > In addition to the required mount option, do you consider these to be a > problem? Do you deal with these "false positives" or do you consider the > result an acceptable margin of error? I do not have data on the error margin. Maybe this can extrapolated from the popcon data. If you would like that only files that are executed (instead of merely read) to be reported, a possible solution is process accounting, see the package GNU acct, but this is more intrusive and probably has an higher overhead than strictatimes. (I never tried it, I would be happy to know more about it.) > My interest comes from working on a similar project. I had the same > problems and came up with a complicated solution. However, if you > already solved the problem, then the only reason to continue working on > my project is it's not necessary to get atime to achieve the end-result. > Albeit, that in itself may not be enough to continue my project since > somebody wishing to use Popcon can simply enable the required mount > option (at some performance cost, I suppose). Using strictatimes has indeed a performance cost (which depend on the filesystem, the hard-disk, etc.) but it should be compared to the performance cost of the alternatives. Any project of this type is a compromise between conflicting goal. For popularity-contest, the goal was to minimize footprint to avoid discouraging users to install it and to avoid popularity-contest itself changing the atimes of too many files. In particular it is written in basic perl (even networking) because all Debian systems are required to provide a basic perl environment. For example, using curl instead of popcon-submit would cause popularity-contest to report curl as used by all submitters. If you have different requirement, you will likely do different choices. Cheers, -- Bill. Imagine a large red swirl here. From jeff at linuxwest.com Fri Mar 4 01:17:05 2011 From: jeff at linuxwest.com (jeff at linuxwest.com) Date: Thu, 03 Mar 2011 18:17:05 -0700 Subject: [Popcon-developers] =?utf-8?q?status_of_executables=3F?= Message-ID: <20110303181705.f50467801c0b453a6c7e53a62af6edb5.c918bc7cce.wbe@email03.secureserver.net> -------- Original Message -------- Subject: Re: [Popcon-developers] status of executables? From: Bill Allombert Date: Thu, March 03, 2011 2:24 pm To: popcon-developers at lists.alioth.debian.org Cc: jeff at linuxwest.com On Wed, Mar 02, 2011 at 04:54:03PM -0700, jeff at linuxwest.com wrote: > Thank you for replying so quick. > > Indeed, mounting with strictatime did yield the desired result. I can > see how some "false positives" *could* happen: > > 1) Opening a directory with a GUI file manager changes atime (to obtain > file types). > 2) Executing `file` on a file. Same as above. > 3) Doing a `grep foo *` within a directory opens the file. > > In addition to the required mount option, do you consider these to be a > problem? Do you deal with these "false positives" or do you consider the > result an acceptable margin of error? I do not have data on the error margin. Maybe this can extrapolated from the popcon data. If you would like that only files that are executed (instead of merely read) to be reported, a possible solution is process accounting, see the package GNU acct, but this is more intrusive and probably has an higher overhead than strictatimes. (I never tried it, I would be happy to know more about it.) > My interest comes from working on a similar project. I had the same > problems and came up with a complicated solution. However, if you > already solved the problem, then the only reason to continue working on > my project is it's not necessary to get atime to achieve the end-result. > Albeit, that in itself may not be enough to continue my project since > somebody wishing to use Popcon can simply enable the required mount > option (at some performance cost, I suppose). Using strictatimes has indeed a performance cost (which depend on the filesystem, the hard-disk, etc.) but it should be compared to the performance cost of the alternatives. Any project of this type is a compromise between conflicting goal. For popularity-contest, the goal was to minimize footprint to avoid discouraging users to install it and to avoid popularity-contest itself changing the atimes of too many files. In particular it is written in basic perl (even networking) because all Debian systems are required to provide a basic perl environment. For example, using curl instead of popcon-submit would cause popularity-contest to report curl as used by all submitters. If you have different requirement, you will likely do different choices. Cheers, -- Bill. Imagine a large red swirl here. ~~ Bill, I understand you completely. My solution uses dnotify and as I said, is perhaps grossly complex. The obstacles are the same and in the end, I don't know if it would offer many overall improvements (of course I would need to run tests). With that said, it is very interesting. Ultimately, your program is much needed for people who want to upgrade their distro regularly. Kudos to you and your team. Jeff From vagrant at debian.org Sun Mar 6 18:54:54 2011 From: vagrant at debian.org (Vagrant Cascadian) Date: Sun, 6 Mar 2011 10:54:54 -0800 Subject: [Popcon-developers] Bug#610876: popularity-contest: Popularity contest will not be run in certain machines due to the DAY variable In-Reply-To: <20110123163508.7330.72160.reportbug@Nox-Perpetua> References: <20110123163508.7330.72160.reportbug@Nox-Perpetua> Message-ID: <20110306185454.GZ10855@talon.fglan> tags 610876 patch thanks On Sun, Jan 23, 2011 at 05:35:08PM +0100, Alberto Aparici wrote: > Popularity-contest is intended to be run approximately once a week. Its ...snip... > But even this has a problem: some computers may not run 7 days > a week. A computer that only runs, for example, on weekends, and ends > up with DAY="3" will never run the popularity contest, not even with > anacron, because anacron will execute the popularity-contest script > but it will exit without running the contest, as it's not Wednesday :-( here's a patch that checks the last time /var/log/popularity-contest was updated, and exits if it was updated more recently than a week ago: diff -U 5 debian/cron.daily.orig debian/cron.daily --- debian/cron.daily.orig 2011-03-06 08:26:52.000000000 -0800 +++ debian/cron.daily 2011-03-06 08:57:43.000000000 -0800 @@ -29,15 +29,25 @@ # Only run on the given day, to spread the load on the server a bit if [ "$DAY" ] && [ "$DAY" != "$(date +%w)" ] ; then exit 0 fi +POPCON=/var/log/popularity-contest +if [ -z "$DAY" ] && [ -f "$POPCON" ] ; then + now=$(date +%s) + lastrun=$(date -r $POPCON +%s) + # one week, in seconds + week=604800 + if [ "$(( $now - $lastrun ))" -le "$week" ]; then + exit 0 + fi +fi + # keep old logs cd /var/log umask 022 savelog -c 7 popularity-contest >/dev/null -POPCON=/var/log/popularity-contest run_popcon() { su -s /bin/sh -c "/usr/sbin/popularity-contest" nobody } it relies on the un-setting DAY in /etc/popularity-contest.conf, though maybe it would make sense to make it the default (by not setting DAY= in postinst)? that would make the first time cron.daily runs the effective day of week to run popularity-contest, rather than randomly selecting a day, while working reasonably sanely on machines that aren't always on. live well, vagrant From owner at bugs.debian.org Sun Mar 6 18:57:06 2011 From: owner at bugs.debian.org (Debian Bug Tracking System) Date: Sun, 06 Mar 2011 18:57:06 +0000 Subject: [Popcon-developers] Processed: Re: popularity-contest: Popularity contest will not be run in certain machines due to the DAY variable In-Reply-To: <20110306185454.GZ10855@talon.fglan> References: <20110306185454.GZ10855@talon.fglan> Message-ID: Processing commands for control at bugs.debian.org: > tags 610876 patch Bug #610876 [popularity-contest] popularity-contest: Popularity contest will not be run in certain machines due to the DAY variable Added tag(s) patch. > thanks Stopping processing here. Please contact me if you need assistance. -- 610876: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=610876 Debian Bug Tracking System Contact owner at bugs.debian.org with problems From Bill.Allombert at math.u-bordeaux1.fr Mon Mar 7 11:33:16 2011 From: Bill.Allombert at math.u-bordeaux1.fr (Bill Allombert) Date: Mon, 7 Mar 2011 12:33:16 +0100 Subject: [Popcon-developers] Bug#610876: popularity-contest: Popularity contest will not be run in certain machines due to the DAY variable In-Reply-To: <20110306185454.GZ10855@talon.fglan> References: <20110123163508.7330.72160.reportbug@Nox-Perpetua> <20110306185454.GZ10855@talon.fglan> Message-ID: <20110307113316.GB13908@yellowpig> On Sun, Mar 06, 2011 at 10:54:54AM -0800, Vagrant Cascadian wrote: > tags 610876 patch > thanks > > On Sun, Jan 23, 2011 at 05:35:08PM +0100, Alberto Aparici wrote: > > Popularity-contest is intended to be run approximately once a week. Its > ...snip... > > But even this has a problem: some computers may not run 7 days > > a week. A computer that only runs, for example, on weekends, and ends > > up with DAY="3" will never run the popularity contest, not even with > > anacron, because anacron will execute the popularity-contest script > > but it will exit without running the contest, as it's not Wednesday :-( > > here's a patch that checks the last time /var/log/popularity-contest was > updated, and exits if it was updated more recently than a week ago: > > diff -U 5 debian/cron.daily.orig debian/cron.daily > --- debian/cron.daily.orig 2011-03-06 08:26:52.000000000 -0800 > +++ debian/cron.daily 2011-03-06 08:57:43.000000000 -0800 > @@ -29,15 +29,25 @@ > # Only run on the given day, to spread the load on the server a bit > if [ "$DAY" ] && [ "$DAY" != "$(date +%w)" ] ; then > exit 0 > fi > > +POPCON=/var/log/popularity-contest > +if [ -z "$DAY" ] && [ -f "$POPCON" ] ; then > + now=$(date +%s) > + lastrun=$(date -r $POPCON +%s) > + # one week, in seconds > + week=604800 > + if [ "$(( $now - $lastrun ))" -le "$week" ]; then > + exit 0 > + fi > +fi > + > # keep old logs > cd /var/log > umask 022 > savelog -c 7 popularity-contest >/dev/null > -POPCON=/var/log/popularity-contest > > run_popcon() > { > su -s /bin/sh -c "/usr/sbin/popularity-contest" nobody > } > Is your patch valid with /bin/sh being dash ? > it relies on the un-setting DAY in /etc/popularity-contest.conf, though maybe > it would make sense to make it the default (by not setting DAY= in postinst)? > that would make the first time cron.daily runs the effective day of week to run > popularity-contest, rather than randomly selecting a day, while working > reasonably sanely on machines that aren't always on. Hello Vagrant, This is an interesting suggestion. Maybe we would change the script to: 1) always try to report on DAY. 2) always try to report if /var/log/popularity-contest is more that 7 days old. Using the first time cron.daily runs as the reporting day unsufficiently random, especially with clusters. A secondary issue is network access. If the script only report through HTTP, it also needs network access to work, so if network is down we could retry the next day. Cheers, -- Bill. Imagine a large red swirl here. From vagrant at debian.org Sat Mar 12 17:27:22 2011 From: vagrant at debian.org (Vagrant Cascadian) Date: Sat, 12 Mar 2011 09:27:22 -0800 Subject: [Popcon-developers] Bug#610876: popularity-contest: Popularity contest will not be run in certain machines due to the DAY variable In-Reply-To: <20110307113316.GB13908@yellowpig> References: <20110123163508.7330.72160.reportbug@Nox-Perpetua> <20110306185454.GZ10855@talon.fglan> <20110307113316.GB13908@yellowpig> Message-ID: <20110312172722.GH10855@talon.fglan> On Mon, Mar 07, 2011 at 12:33:16PM +0100, Bill Allombert wrote: > Is your patch valid with /bin/sh being dash ? i don't see anything that would be a problem... i haven't actually tested the code, but i've used all the constructs in dash scripts in the past without problem. is there something specific that you're wondering about? > This is an interesting suggestion. Maybe we would change the script to: > 1) always try to report on DAY. > 2) always try to report if /var/log/popularity-contest is more that 7 days old. > > Using the first time cron.daily runs as the reporting day unsufficiently > random, especially with clusters. the following updated patch should mostly address those concerns (and an inverted less-than/greater-than issue). it will cause it to run once on the first cron.daily run after install, but after that respect the DAY variable: --- cron.daily.orig 2011-03-06 08:26:52.000000000 -0800 +++ cron.daily 2011-03-12 08:52:31.000000000 -0800 @@ -26,16 +26,26 @@ export http_proxy="$HTTP_PROXY"; fi +POPCON=/var/log/popularity-contest + # Only run on the given day, to spread the load on the server a bit if [ "$DAY" ] && [ "$DAY" != "$(date +%w)" ] ; then - exit 0 + # Ensure that popcon runs at least once in the last week + if [ -f "$POPCON" ] ; then + now=$(date +%s) + lastrun=$(date -r $POPCON +%s) + # one week, in seconds + week=604800 + if [ "$(( $now - $lastrun ))" -ge "$week" ]; then + exit 0 + fi + fi fi # keep old logs cd /var/log umask 022 savelog -c 7 popularity-contest >/dev/null -POPCON=/var/log/popularity-contest run_popcon() { an alternate approach would be to simply touch $POPCON on the first cron.daily run, though you might miss the first week of popcon data if it happens to be offline or without network on $DAY: --- cron.daily.orig 2011-03-06 08:26:52.000000000 -0800 +++ cron.daily 2011-03-12 09:02:00.000000000 -0800 @@ -26,16 +26,29 @@ export http_proxy="$HTTP_PROXY"; fi +POPCON=/var/log/popularity-contest + # Only run on the given day, to spread the load on the server a bit if [ "$DAY" ] && [ "$DAY" != "$(date +%w)" ] ; then - exit 0 + # Ensure that popcon runs at least once in the last week + if [ -f "$POPCON" ] ; then + now=$(date +%s) + lastrun=$(date -r $POPCON +%s) + # one week, in seconds + week=604800 + if [ "$(( $now - $lastrun ))" -ge "$week" ]; then + exit 0 + fi + else + touch $POPCON + exit 0 + fi fi # keep old logs cd /var/log umask 022 savelog -c 7 popularity-contest >/dev/null -POPCON=/var/log/popularity-contest run_popcon() { could probably come up with something more complicated that handles the corner cases more perfectly, but it might not be worth the code complexity. both patches might need some fiddling with the value of week in seconds, since simply running popcon might take longer than a second; you might end up consistantly running every 8 days (for machines that don't typically run on $DAY). subtracting a few hours (5-12?) from the week variable might be the simplest way to handle that. i wish i had followed up on this idea in time for lenny (yes, lenny) release... heh. :) live well, vagrant From vagrant at freegeek.org Mon Mar 14 18:54:22 2011 From: vagrant at freegeek.org (Vagrant Cascadian) Date: Mon, 14 Mar 2011 11:54:22 -0700 Subject: [Popcon-developers] Bug#610876: popularity-contest: Popularity contest will not be run in certain machines due to the DAY variable In-Reply-To: <20110312172722.GH10855@talon.fglan> References: <20110123163508.7330.72160.reportbug@Nox-Perpetua> <20110306185454.GZ10855@talon.fglan> <20110307113316.GB13908@yellowpig> <20110312172722.GH10855@talon.fglan> Message-ID: <20110314185422.GL10855@talon.fglan> On Sat, Mar 12, 2011 at 09:27:22AM -0800, Vagrant Cascadian wrote: > On Mon, Mar 07, 2011 at 12:33:16PM +0100, Bill Allombert wrote: > > Is your patch valid with /bin/sh being dash ? > > i don't see anything that would be a problem... i haven't actually tested the > code, but i've used all the constructs in dash scripts in the past without > problem. is there something specific that you're wondering about? i've now tested the following patch with both bash and dash, and it appears to work fine. > > This is an interesting suggestion. Maybe we would change the script to: > > 1) always try to report on DAY. > > 2) always try to report if /var/log/popularity-contest is more that 7 days old. > > > > Using the first time cron.daily runs as the reporting day unsufficiently > > random, especially with clusters. ok, i've actually tested this patch a bit (my original -le was right, gah): --- cron.daily.orig 2011-03-06 08:26:52.000000000 -0800 +++ /etc/cron.daily/popularity-contest 2011-03-14 11:11:43.000000000 -0700 @@ -26,16 +26,30 @@ export http_proxy="$HTTP_PROXY"; fi +POPCON=/var/log/popularity-contest + # Only run on the given day, to spread the load on the server a bit if [ "$DAY" ] && [ "$DAY" != "$(date +%w)" ] ; then - exit 0 + # Ensure that popcon runs at least once in the last week + if [ -f "$POPCON" ] ; then + now=$(date +%s) + lastrun=$(date -r $POPCON +%s) + # one week (minus 5 hours), in seconds + # $((60*60*24*7-60*60*5)) + week=586800 + if [ "$(( $now - $lastrun ))" -le "$week" ]; then + exit 0 + fi + else + touch $POPCON + exit 0 + fi fi # keep old logs cd /var/log umask 022 savelog -c 7 popularity-contest >/dev/null -POPCON=/var/log/popularity-contest run_popcon() { > both patches might need some fiddling with the value of week in seconds, since > simply running popcon might take longer than a second; you might end up > consistantly running every 8 days (for machines that don't typically run on > $DAY). subtracting a few hours (5-12?) from the week variable might be the > simplest way to handle that. i just picked 5 hours. easy enough to adjust, but i think that should align most cases on the weekly schedule. i also went with the patch that creates an empty /var/log/popularity-contest on the first run, which may result in missing the first week's results if it doesn't happen to run on $DAY. no big loss, in the grand scheme of things. wasn't sure weather to use tabs or spaces, it seemed like most of the code was using tabs, but not consistantly. live well, vagrant From owner at bugs.debian.org Tue Mar 22 12:51:11 2011 From: owner at bugs.debian.org (Debian Bug Tracking System) Date: Tue, 22 Mar 2011 12:51:11 +0000 Subject: [Popcon-developers] Processed: limit source to popularity-contest, tagging 614694, tagging 610876 In-Reply-To: <1300798218-3101-bts-ballombe@debian.org> References: <1300798218-3101-bts-ballombe@debian.org> Message-ID: Processing commands for control at bugs.debian.org: > #popularity-contest (1.52) UNRELEASED; urgency=low > # > # * Translations: > # - Nepali (Shyam Krishna Bal). Closes: #614694 > # * Allow cron.daily to run if last run was more than 7 day ago. > # Patch from Vagrant Cascadian. Closes: #610876 > # > limit source popularity-contest Limiting to bugs with field 'source' containing at least one of 'popularity-contest' Limit currently set to 'source':'popularity-contest' > tags 614694 + pending Bug #614694 [popularity-contest] [INTL:ne] Popularity-contest eject translation Added tag(s) pending. > tags 610876 + pending Bug #610876 [popularity-contest] popularity-contest: Popularity contest will not be run in certain machines due to the DAY variable Added tag(s) pending. > thanks Stopping processing here. Please contact me if you need assistance. -- 610876: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=610876 614694: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=614694 Debian Bug Tracking System Contact owner at bugs.debian.org with problems From Bill.Allombert at math.u-bordeaux1.fr Tue Mar 22 13:45:42 2011 From: Bill.Allombert at math.u-bordeaux1.fr (Bill Allombert) Date: Tue, 22 Mar 2011 14:45:42 +0100 Subject: [Popcon-developers] Bug#502159: Bug#502159: Bug#502159: popularity-contest: when HTTP fails, tries MAILTO: no dpkg-configure options In-Reply-To: <20081013.214017.93455921.furue@hawaii.edu> References: <20081013.171331.199038656.furue@hawaii.edu> <20081014071445.GE21511@klodrik.uio.no> <20081013.214017.93455921.furue@hawaii.edu> Message-ID: <20110322134542.GC9904@yellowpig> On Mon, Oct 13, 2008 at 09:40:17PM -1000, Ryo Furue wrote: > Petter, > > Thank you for your response. > > | > I think this behavior should be configurable through > | > "dpkg-reconfigure popularity-contest", because > | > > | > 1) There are many users whose MTA only allows local delivery; > | > 2) How to change configuration isn't clear. > | > | The correct way is to add a line with > | > | MAILTO= > | > | in /etc/popularity-contest.conf, to override the default email > | address. > > Thanks. I was actually able to figure that out by reading > /etc/cron.daily/popularity-contest . > > | I agree that this should be mentioned in the manual page, at > | the very least. :) I suggest that the attached patch be applied to the FAQ. This replace an outdated question by the following: - - - - - - - - - - - Q) How can I prevent popularity-contest to send report via email ? A) This is not recommended. Report are sent by email only when the HTTP submission fails which is generally caused by a temporary lack of internet connectivity. By contrast, reports sent by email are stored in the mail server queue until the internet connectivity is back. Nevertheless, you can prevent popularity-contest to send report via email by adding MAILTO= to /etc/popularity-contest.conf - - - - - - - - - - - Cheers, -- Bill. Imagine a large red swirl here. -------------- next part -------------- Index: FAQ =================================================================== --- FAQ (r?vision 637) +++ FAQ (copie de travail) @@ -75,15 +75,16 @@ su myuser -s /bin/sh -c "/usr/sbin/sendmail -oi \"$MAILTO\"" } -Q) My system is unable to send email out to the Internet. How can I -participate ? +Q) How can I prevent popularity-contest to send report via email ? -A) Since version 1.30 of popularity-contest, it is possible to use HTTP -as the transport protocol. This is the default for new installations, -but needs to be manually enabled for old installations. To enable it, -run this commmand as root: +A) This is not recommended. Report are sent by email only when the HTTP +submission fails which is generally caused by a temporary lack of internet +connectivity. By contrast, reports sent by email are stored in the mail server +queue until the internet connectivity is back. - dpkg-reconfigure popularity-contest +Nevertheless, you can prevent popularity-contest to send report via email by adding +MAILTO= +to /etc/popularity-contest.conf Q) How can I specify a HTTP proxy server for the HTTP submission ? From owner at bugs.debian.org Tue Mar 22 14:45:07 2011 From: owner at bugs.debian.org (Debian Bug Tracking System) Date: Tue, 22 Mar 2011 14:45:07 +0000 Subject: [Popcon-developers] Bug#576963: marked as done ([popularity-contest] http://popcon.debian.org/all-popcon-results.txt.gz 2x compressed) References: <20110322144332.GD9904@yellowpig> <4BBDF298.2040403@debian.org> Message-ID: Your message dated Tue, 22 Mar 2011 15:43:32 +0100 with message-id <20110322144332.GD9904 at yellowpig> and subject line Re: Bug#576963: [popularity-contest] http://popcon.debian.org/all-popcon-results.txt.gz 2x compressed has caused the Debian Bug report #576963, regarding [popularity-contest] http://popcon.debian.org/all-popcon-results.txt.gz 2x compressed to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact owner at bugs.debian.org immediately.) -- 576963: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=576963 Debian Bug Tracking System Contact owner at bugs.debian.org with problems -------------- next part -------------- An embedded message was scrubbed... From: Bastian Venthur Subject: [popularity-contest] http://popcon.debian.org/all-popcon-results.txt.gz 2x compressed Date: Thu, 08 Apr 2010 17:13:28 +0200 Size: 3415 URL: -------------- next part -------------- An embedded message was scrubbed... From: Bill Allombert Subject: Re: Bug#576963: [popularity-contest] http://popcon.debian.org/all-popcon-results.txt.gz 2x compressed Date: Tue, 22 Mar 2011 15:43:32 +0100 Size: 3557 URL: From tiago at debian.org Wed Mar 23 01:27:34 2011 From: tiago at debian.org (Tiago Bortoletto Vaz) Date: Tue, 22 Mar 2011 22:27:34 -0300 Subject: [Popcon-developers] Accessing popcon data Message-ID: <20110323012734.GA15828@debian.org> Hi, I'm sponsoring T?ssia in her Master's project, which intends to be useful (as truly usable) for Debian in the future: "The main goal of my master's project is to develop an application recommender for GNU/Linux distributions, based on the system installed applications and other systems' behaviour (for colaborative strategies)." http://www.ime.usp.br/~tassia/apprec.html We've reached the point where we need to perform some popcon raw data processing in order to get a realistic status of the strategies coded. I'm fully aware of privacy issues involved and hereby state that any raw data or resulted information from it will be shared/published to noone/nowhere without discussion (and no objection) in popcon-developers list. We actually intend to provide some general statistics in the future. However, every step regarding popcon data for this project will be presented here before making results public. T?ssia is a long term Debian user and contributor from Brazil. I'm sure she is very trusted by many other Debian Developers, mainly due to her work in past (and current) DebConfs. She is also my wife - if that matters here. That said, if there's no objection, I'll go ahead in few days. ps: Cc'ing Enrico and Zack, once they have recently been interacting with T?ssia on this. Best regards, -- -------------------------------------------------------------------------------- .''`. Tiago Bortoletto Vaz GPG : 1024D/A504FECA : :' : http://tiagovaz.org XMPP : tiago at jabber.org `. `' tiago at {tiagovaz,debian}.org IRC : tiago at OFTC `- Debian GNU/Linux - The Universal OS http://www.debian.org -------------------------------------------------------------------------------- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From Bill.Allombert at math.u-bordeaux1.fr Wed Mar 23 12:55:15 2011 From: Bill.Allombert at math.u-bordeaux1.fr (Bill Allombert) Date: Wed, 23 Mar 2011 13:55:15 +0100 Subject: [Popcon-developers] [Debian RT] popcon broken: bellini NFS link for /srv/ftp.debian.org/ftp unreliable Message-ID: <20110323125515.GD12258@yellowpig> Dear Debian system admins, Since the 16th of March, the popcon cron job cannot complete because it cannot access the Packages.gz files below /org/ftp.debian.org/ftp. On bellini.debian.org, this directory is NFS mounted. This cronjob is run as the popcon user, and it seems there are permission issues that prevent the popcon user to access the NFS share. For example running ls as the ballombe user work: bellini% ls /org/ftp.debian.org/ftp dists indices ls-lR.patch.gz project README.CD-manufacture README.mirrors.html tools doc ls-lR.gz pool README README.html README.mirrors.txt but it fails when running as the popcon user: bellini% sudo -u popcon ls /org/ftp.debian.org/ftp As the result the popcon report cannot be computed and the popcon.debian.org website cannot be updated. Cheers, -- Bill. Imagine a large red swirl here. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From Bill.Allombert at math.u-bordeaux1.fr Wed Mar 23 14:58:15 2011 From: Bill.Allombert at math.u-bordeaux1.fr (Bill Allombert) Date: Wed, 23 Mar 2011 15:58:15 +0100 Subject: [Popcon-developers] Accessing popcon data In-Reply-To: <20110323012734.GA15828@debian.org> References: <20110323012734.GA15828@debian.org> Message-ID: <20110323145815.GE12258@yellowpig> On Tue, Mar 22, 2011 at 10:27:34PM -0300, Tiago Bortoletto Vaz wrote: > Hi, > > I'm sponsoring T?ssia in her Master's project, which intends to be useful (as > truly usable) for Debian in the future: > > "The main goal of my master's project is to develop an application recommender > for GNU/Linux distributions, based on the system installed applications and > other systems' behaviour (for colaborative strategies)." > > http://www.ime.usp.br/~tassia/apprec.html > > We've reached the point where we need to perform some popcon raw data > processing in order to get a realistic status of the strategies coded. > > I'm fully aware of privacy issues involved and hereby state that any raw data > or resulted information from it will be shared/published to noone/nowhere > without discussion (and no objection) in popcon-developers list. We actually > intend to provide some general statistics in the future. However, every step > regarding popcon data for this project will be presented here before making > results public. > > T?ssia is a long term Debian user and contributor from Brazil. I'm sure she is > very trusted by many other Debian Developers, mainly due to her work in past > (and current) DebConfs. She is also my wife - if that matters here. Hello Tiago, Our policy is that all Debian developpers should have access to the data and they are free to use them as long as they understand the privacy policy and they take responsibility for any information disclosure. In short, I have no objection with your project. Now, it would be great if there was some research on how much information a recommendation system leaks, and how that can be mitigated. A whole new master project I am afraid. Cheers, -- Bill. Imagine a large red swirl here. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From tassia at gmail.com Wed Mar 23 16:50:44 2011 From: tassia at gmail.com (=?ISO-8859-1?Q?T=E1ssia_Cam=F5es?=) Date: Wed, 23 Mar 2011 13:50:44 -0300 Subject: [Popcon-developers] Accessing popcon data In-Reply-To: <20110323145815.GE12258@yellowpig> References: <20110323012734.GA15828@debian.org> <20110323145815.GE12258@yellowpig> Message-ID: Hello Bill, 2011/3/23 Bill Allombert : > > Now, it would be great if there was some research on how much information a recommendation > system leaks, and how that can be mitigated. A whole new master project I am afraid. I plan to bring this questions in my work, but I'll probably not go too deep since my main goal is to develop the recommender. Indeed, there are some researches in this field. As an example, there is this article from researchers of University of Texas at Austin which case study discusses how to break anonymity of the Netflix prize dataset [1]. And in fact the contest was canceled after a privacy lawsuit. However I don't see these privacy issues as a problem of the recommendation systems themselves, these leaks are all consequence of the disclosure of the transactions database. Netflix had to do it for the contest, since the participants could not develop a solution without having access to the database. And so they got suited. In this project, what will be available is the recommender, not the database. The recommendation is a result of processing the whole bunch of data to give suggestions to a specific user based on commonalities of behavior. I can't see how individual records could be tracked if you only have access to the recommendation. Do you? Regards, T?ssia. [1] Arvind Narayanan and Vitaly Shmatikov, Robust De-anonymization of Large Sparse Datasets, http://www.cs.utexas.edu/~shmat/shmat_oak08netflix.pdf [2] http://www.wired.com/threatlevel/2010/03/netflix-cancels-contest/ From Bill.Allombert at math.u-bordeaux1.fr Wed Mar 23 18:17:18 2011 From: Bill.Allombert at math.u-bordeaux1.fr (Bill Allombert) Date: Wed, 23 Mar 2011 19:17:18 +0100 Subject: [Popcon-developers] Accessing popcon data In-Reply-To: References: <20110323012734.GA15828@debian.org> <20110323145815.GE12258@yellowpig> Message-ID: <20110323181717.GA20558@yellowpig> On Wed, Mar 23, 2011 at 01:50:44PM -0300, T?ssia Cam?es wrote: > Hello Bill, > > 2011/3/23 Bill Allombert : > > > > Now, it would be great if there was some research on how much information a recommendation > > system leaks, and how that can be mitigated. A whole new master project I am afraid. > > I plan to bring this questions in my work, but I'll probably not go > too deep since my main goal is to develop the recommender. I understand that! > Indeed, there are some researches in this field. As an example, there > is this article from researchers of University of Texas at Austin > which case study discusses how to break anonymity of the Netflix prize > dataset [1]. And in fact the contest was canceled after a privacy > lawsuit. > > However I don't see these privacy issues as a problem of the > recommendation systems themselves, these leaks are all consequence of > the disclosure of the transactions database. > Netflix had to do it for the contest, since the participants could not > develop a solution without having access to the database. And so they > got suited. > > In this project, what will be available is the recommender, not the > database. The recommendation is a result of processing the whole bunch > of data to give suggestions to a specific user based on commonalities > of behavior. I can't see how individual records could be tracked if > you only have access to the recommendation. Do you? I will come to that later. But for now, I like to define a leak as the possibility to guess information about the data that is not provided by the current popcon.debian.org. Let consider D to be the data set you are using, R the recommender, q a query and C the recommendation. We have the equation R(D,q)=C. Now assume that someone has access to the code source of R and is able to make an infinite number of query to R. They can ask themself: what are all the possible datasets D such that R(D,q)=C for all query ? The less datasets are possible, the more information is leaked. More realisticaly, they can make an hypothesis H and check its validity by making query. Since they have access to the algorithm used by the recommender, they can infer some information from the answer. For example, they might know that user X has packages foo1, foo2, foo3 installed, and they want to know whether it is likely that X has installed bar1, bar2 or bar3. So they might submit queries to the recommender and see whether it suggests bar1, bar2 or bar3. Or they have found a security issue in package bar, and are looking at server running bar. They can search web servers that run such and such webservice packages (for example they can check whether the server run the Debian version of apache, etc) and they might use the recommender to find a set of webservices that is correlated to the installation of the package bar. Theses are passive attacks. But of course, they could submit hand-crafted popcon reports to affect the dataset and change the behaviour of the recommender to increase their chance of success. Now, to the subject of tracking individual records: it depends how the recommender works, but some packages are only installed by a single user. Sometime the package is even private to that user (e.g. home-built kernels with custom version). This information could be used to guess other packages installed by the same user. Cheers, -- Bill. Imagine a large red swirl here. From tassia at gmail.com Wed Mar 23 20:06:51 2011 From: tassia at gmail.com (=?ISO-8859-1?Q?T=E1ssia_Cam=F5es?=) Date: Wed, 23 Mar 2011 17:06:51 -0300 Subject: [Popcon-developers] Accessing popcon data In-Reply-To: <20110323181717.GA20558@yellowpig> References: <20110323012734.GA15828@debian.org> <20110323145815.GE12258@yellowpig> <20110323181717.GA20558@yellowpig> Message-ID: Hi Bill, Thanks for your inputs about this, they are very much appreciated! 2011/3/23 Bill Allombert : > > I will come to that later. But for now, I like to define a leak as the > possibility to guess information about the data that is not provided by > the current popcon.debian.org. > In fact I was only worried about avoiding privacy leaks to ensure users privacy. I don't get what is the problem of providing more information than what is currently provided by popcon reports (except that it should be agreed here first), given that we safeguard users individual records. Don't you think it could contribute to make Debian better? If we manage to develop an accurate enough recommender such a system would help users to meet potentially desired packages. Assuming that it would increase package popularity in general, we would be improving Debian (more users, more bug reports, more contributors etc). > Let consider D to be the data set you are using, R the > recommender, q a query and C the recommendation. > > We have the equation R(D,q)=C. > > Now assume that someone has access to the code source of R and is able to make > an infinite number of query to R. They can ask themself: what are all the > possible datasets D such that R(D,q)=C for all query ? > The less datasets are possible, the more information is leaked. > > For example, they might know that user X has packages foo1, foo2, foo3 installed, > and they want to know whether it is likely that X has installed bar1, bar2 or bar3. > So they might submit queries to the recommender and see whether it suggests > bar1, bar2 or bar3. > I agree with you, but again, I don't think this kind of association is a problem. I believe quite the opposite actually. I think it might be useful to know that some packages are usually together even though they are not dependent from each other. This kind of info might be useful for QA, for example. > Or they have found a security issue in package bar, and are looking at server running > bar. They can search web servers that run such and such webservice packages (for example they > can check whether the server run the Debian version of apache, etc) and they might use > the recommender to find a set of webservices that is correlated to the installation of the > package bar. > > Theses are passive attacks. But of course, they could submit hand-crafted popcon reports > to affect the dataset and change the behaviour of the recommender to increase their chance > of success. > This is a central issue about data mining. When we help people to better understand a collection of data we let them free to use this knowledge as they want. And in fact this could be extrapoled to any field of science. You never know if what you are researching/developing will be used for evil things. It's a pitty. > Now, to the subject of tracking individual records: it depends how the recommender works, > but some packages are only installed by a single user. Sometime the package is even private > to that user (e.g. home-built kernels with custom version). This information could be used > to guess other packages installed by the same user. > Regarding this point I'd say that private packages and those with very few or a single user would be totally low priority for recommendations, and could even be ignored. [ ]'s T?ssia. From enrico at debian.org Wed Mar 23 21:47:24 2011 From: enrico at debian.org (Enrico Zini) Date: Wed, 23 Mar 2011 21:47:24 +0000 Subject: [Popcon-developers] Accessing popcon data In-Reply-To: References: <20110323012734.GA15828@debian.org> <20110323145815.GE12258@yellowpig> Message-ID: <20110323214724.GA27280@enricozini.org> On Wed, Mar 23, 2011 at 01:50:44PM -0300, T?ssia Cam?es wrote: > In this project, what will be available is the recommender, not the > database. The recommendation is a result of processing the whole bunch > of data to give suggestions to a specific user based on commonalities > of behavior. I can't see how individual records could be tracked if > you only have access to the recommendation. Do you? Here is an example: pick a package X which has '1' inst on popcon; note who is the maintainer. Then query a recommender asking 'what do you suggest me to install if I have package X installed?': chances are that, with many recommenders, the result is a list of the packages installed on the maintainer's machine. This specific example attack can be prevented by ignoring packages installed in less that a certain number of systems, but there can be more examples. It is of course not the point of your research, and as long as there are no obvious ways to exploit the recommender, I don't think the lack of literature on information leaks will be an obstacle for deployment once it works. Ciao, Enrico -- GPG key: 4096R/E7AD5568 2009-05-08 Enrico Zini -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From tassia at gmail.com Thu Mar 24 15:49:36 2011 From: tassia at gmail.com (=?ISO-8859-1?Q?T=E1ssia_Cam=F5es?=) Date: Thu, 24 Mar 2011 12:49:36 -0300 Subject: [Popcon-developers] Accessing popcon data In-Reply-To: <20110323214724.GA27280@enricozini.org> References: <20110323012734.GA15828@debian.org> <20110323145815.GE12258@yellowpig> <20110323214724.GA27280@enricozini.org> Message-ID: Thanks for your inputs enrico. 2011/3/23 Enrico Zini : > > Here is an example: pick a package X which has '1' inst on popcon; note > who is the maintainer. Then query a recommender asking 'what do you > suggest me to install if I have package X installed?': chances are that, > with many recommenders, the result is a list of the packages installed > on the maintainer's machine. > > This specific example attack can be prevented by ignoring packages > installed in less that a certain number of systems, but there can be > more examples. > Yes, I'm aware of this one. As I said on the other message, we intend to ignore those packages with very feel installations when computing recommendations. > It is of course not the point of your research, and as long as there are > no obvious ways to exploit the recommender, I don't think the lack of > literature on information leaks will be an obstacle for deployment once > it works. > Great ;-) It's very good to have a discussion on this though. Cheers, T?ssia. From Bill.Allombert at math.u-bordeaux1.fr Wed Mar 30 15:26:53 2011 From: Bill.Allombert at math.u-bordeaux1.fr (Bill Allombert) Date: Wed, 30 Mar 2011 17:26:53 +0200 Subject: [Popcon-developers] Bug#502159: Bug#502159: Bug#502159: popularity-contest: when HTTP fails, tries MAILTO: no dpkg-configure options In-Reply-To: <20110322134542.GC9904@yellowpig> References: <20081013.171331.199038656.furue@hawaii.edu> <20081014071445.GE21511@klodrik.uio.no> <20081013.214017.93455921.furue@hawaii.edu> <20110322134542.GC9904@yellowpig> Message-ID: <20110330152653.GC18485@yellowpig> On Tue, Mar 22, 2011 at 02:45:42PM +0100, Bill Allombert wrote: > I suggest that the attached patch be applied to the FAQ. I have updated http://popcon.debian.org/FAQ and added this entry: Q) How can I prevent popularity-contest from sending reports via email? A) This is not recommended. Reports are sent by email only when the HTTP submission fails, which is generally caused by a temporary lack of internet connectivity. By contrast, reports sent by email are stored in the mail server queue until the internet connectivity is back. Nevertheless, you can prevent popularity-contest from sending reports via email by adding MAILTO= to /etc/popularity-contest.conf Cheers, -- Bill. Imagine a large red swirl here. From Bill.Allombert at math.u-bordeaux1.fr Wed Mar 30 15:30:10 2011 From: Bill.Allombert at math.u-bordeaux1.fr (Bill Allombert) Date: Wed, 30 Mar 2011 17:30:10 +0200 Subject: [Popcon-developers] Bug#610876: popularity-contest: Popularity contest will not be run in certain machines due to the DAY variable In-Reply-To: <20110312172722.GH10855@talon.fglan> References: <20110123163508.7330.72160.reportbug@Nox-Perpetua> <20110306185454.GZ10855@talon.fglan> <20110307113316.GB13908@yellowpig> <20110312172722.GH10855@talon.fglan> Message-ID: <20110330153010.GD18485@yellowpig> On Sat, Mar 12, 2011 at 09:27:22AM -0800, Vagrant Cascadian wrote: > On Mon, Mar 07, 2011 at 12:33:16PM +0100, Bill Allombert wrote: > > Is your patch valid with /bin/sh being dash ? > > i don't see anything that would be a problem... i haven't actually tested the > code, but i've used all the constructs in dash scripts in the past without > problem. is there something specific that you're wondering about? > > > > This is an interesting suggestion. Maybe we would change the script to: > > 1) always try to report on DAY. > > 2) always try to report if /var/log/popularity-contest is more that 7 days old. > > > > Using the first time cron.daily runs as the reporting day unsufficiently > > random, especially with clusters. > > the following updated patch should mostly address those concerns (and an > inverted less-than/greater-than issue). it will cause it to run once on the > first cron.daily run after install, but after that respect the DAY variable: > > --- cron.daily.orig 2011-03-06 08:26:52.000000000 -0800 > +++ cron.daily 2011-03-12 08:52:31.000000000 -0800 > @@ -26,16 +26,26 @@ > export http_proxy="$HTTP_PROXY"; > fi > > +POPCON=/var/log/popularity-contest > + > # Only run on the given day, to spread the load on the server a bit > if [ "$DAY" ] && [ "$DAY" != "$(date +%w)" ] ; then > - exit 0 > + # Ensure that popcon runs at least once in the last week > + if [ -f "$POPCON" ] ; then > + now=$(date +%s) > + lastrun=$(date -r $POPCON +%s) > + # one week, in seconds > + week=604800 > + if [ "$(( $now - $lastrun ))" -ge "$week" ]; then > + exit 0 > + fi > + fi > fi > > # keep old logs > cd /var/log > umask 022 > savelog -c 7 popularity-contest >/dev/null > -POPCON=/var/log/popularity-contest > > run_popcon() > { Hello Vagrant, I have applied your patch to popcon SVN on alioth, thanks! Next time you send two patches in a single email, send one as an attachement, else patch will apply both! Cheers, -- Bill. Imagine a large red swirl here. From Bill.Allombert at math.u-bordeaux1.fr Wed Mar 30 15:46:27 2011 From: Bill.Allombert at math.u-bordeaux1.fr (Bill Allombert) Date: Wed, 30 Mar 2011 17:46:27 +0200 Subject: [Popcon-developers] Bug#610860: popularity-contest: abnormal termination while a package is uninstalled In-Reply-To: <20110123130350.5392.80843.reportbug@mobee> References: <20110123130350.5392.80843.reportbug@mobee> Message-ID: <20110330154627.GE18485@yellowpig> On Sun, Jan 23, 2011 at 02:03:50PM +0100, Serafeim Zanikolas wrote: > Package: popularity-contest > Version: 1.49 > Severity: normal > Tags: patch > > Hi, > > popcon failed with the error below while I've been purging another package: > > /etc/cron.daily/popularity-contest: > readline() on closed filehandle FILES at /usr/sbin/popularity-contest line 104. Hello Serafeim, As far as I understand, this is a warning, not an anormal termination. Yes, we could check the return value of open, but it this instance I feel that the warning is legitimate. For example, if the user set up a cron job that automatically update their system, it can alert them that there is a conflict. Please note that this bug was also reported as #555790. I am still not quite sure how to address it. Cheers, -- Bill. Imagine a large red swirl here. From owner at bugs.debian.org Wed Mar 30 16:42:38 2011 From: owner at bugs.debian.org (Debian Bug Tracking System) Date: Wed, 30 Mar 2011 16:42:38 +0000 Subject: [Popcon-developers] Processed: tagging 610860 In-Reply-To: <1301502408-1393-bts-ballombe@debian.org> References: <1301502408-1393-bts-ballombe@debian.org> Message-ID: Processing commands for control at bugs.debian.org: > tags 610860 - patch Bug #610860 [popularity-contest] popularity-contest: abnormal termination while a package is uninstalled Removed tag(s) patch. > thanks Stopping processing here. Please contact me if you need assistance. -- 610860: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=610860 Debian Bug Tracking System Contact owner at bugs.debian.org with problems From owner at bugs.debian.org Wed Mar 30 16:42:40 2011 From: owner at bugs.debian.org (Debian Bug Tracking System) Date: Wed, 30 Mar 2011 16:42:40 +0000 Subject: [Popcon-developers] Processed: merging 610860 555790 In-Reply-To: <1301501044-2619-bts-ballombe@debian.org> References: <1301501044-2619-bts-ballombe@debian.org> Message-ID: Processing commands for control at bugs.debian.org: > merge 610860 555790 Bug#555790: readline() on closed filehandle FILES at /usr/sbin/popularity-contest line 104. Bug#610860: popularity-contest: abnormal termination while a package is uninstalled Merged 555790 610860. > thanks Stopping processing here. Please contact me if you need assistance. -- 610860: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=610860 555790: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=555790 Debian Bug Tracking System Contact owner at bugs.debian.org with problems From owner at bugs.debian.org Wed Mar 30 16:51:05 2011 From: owner at bugs.debian.org (Debian Bug Tracking System) Date: Wed, 30 Mar 2011 16:51:05 +0000 Subject: [Popcon-developers] Processed: limit source to popularity-contest, tagging 403916, tagging 502159 In-Reply-To: <1301503723-2411-bts-ballombe@debian.org> References: <1301503723-2411-bts-ballombe@debian.org> Message-ID: Processing commands for control at bugs.debian.org: > #popularity-contest (1.52) UNRELEASED; urgency=low > # > # * FAQ: document how to disable sending report by email. Closes: #502159 > # Thanks, Ryo Furue. > # * debian/cron.daily, FAQ: > # - Add variable MTAOPS to pass extra options to sendmail. Closes: #403916. > # Thanks Michelle Konzack > # > limit source popularity-contest Limiting to bugs with field 'source' containing at least one of 'popularity-contest' Limit currently set to 'source':'popularity-contest' > tags 403916 + pending Bug #403916 [popularity-contest] popularity-contest should support MTAOPTS Added tag(s) pending. > tags 502159 + pending Bug #502159 [popularity-contest] popularity-contest: when HTTP fails, tries MAILTO: no dpkg-configure options Added tag(s) pending. > thanks Stopping processing here. Please contact me if you need assistance. -- 502159: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=502159 403916: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=403916 Debian Bug Tracking System Contact owner at bugs.debian.org with problems From owner at bugs.debian.org Wed Mar 30 17:09:12 2011 From: owner at bugs.debian.org (Debian Bug Tracking System) Date: Wed, 30 Mar 2011 17:09:12 +0000 Subject: [Popcon-developers] Bug#577806: marked as done (popularity-contest: Participate is not the default action) References: <20110330163602.GA27249@yellowpig> <20100414190629.2176.71721.reportbug@localhost> Message-ID: Your message dated Wed, 30 Mar 2011 18:36:02 +0200 with message-id <20110330163602.GA27249 at yellowpig> and subject line Re: Bug#577806: popularity-contest: Participate is not the default action has caused the Debian Bug report #577806, regarding popularity-contest: Participate is not the default action to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact owner at bugs.debian.org immediately.) -- 577806: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=577806 Debian Bug Tracking System Contact owner at bugs.debian.org with problems -------------- next part -------------- An embedded message was scrubbed... From: Luca Bruno Subject: popularity-contest: Participate is not the default action Date: Wed, 14 Apr 2010 21:06:29 +0200 Size: 4185 URL: -------------- next part -------------- An embedded message was scrubbed... From: Bill Allombert Subject: Re: Bug#577806: popularity-contest: Participate is not the default action Date: Wed, 30 Mar 2011 18:36:02 +0200 Size: 3954 URL: From sez at debian.org Wed Mar 30 17:45:57 2011 From: sez at debian.org (Serafeim Zanikolas) Date: Wed, 30 Mar 2011 19:45:57 +0200 Subject: [Popcon-developers] Bug#610860: popularity-contest: abnormal termination while a package is uninstalled In-Reply-To: <20110330154627.GE18485@yellowpig> References: <20110123130350.5392.80843.reportbug@mobee> <20110330154627.GE18485@yellowpig> Message-ID: <20110330174557.GA23664@mobee> Hi Bill, On Wed, Mar 30, 2011 at 05:46:27PM +0200, Bill Allombert wrote [edited]: > As far as I understand, this is a warning, not an anormal termination. > Yes, we could check the return value of open, but it this instance I > feel that the warning is legitimate. Agreed, but the user that sees the warning cannot tell that. > For example, if the user set up a cron job that automatically update their system, > it can alert them that there is a conflict. > > Please note that this bug was also reported as #555790. I am still not quite sure > how to address it. Perhaps add a msg to make it clear that it is normal under certain circumstances (along the lines of "failed to open xxx (this is normal if dpkg was running at the same time)". This will save you from having to deal with such bug reports in the future. I would also consider to completely silence the msg when popcon runs via cron. I don't want to get an email if it "fails" once a year, just because of a coincidence for which I will do nothing about. Cheers, Serafeim -- Every great idea is worthless without someone to do the work. --Neil Williams From Bill.Allombert at math.u-bordeaux1.fr Wed Mar 30 20:46:59 2011 From: Bill.Allombert at math.u-bordeaux1.fr (Bill Allombert) Date: Wed, 30 Mar 2011 22:46:59 +0200 Subject: [Popcon-developers] Bug#610860: popularity-contest: abnormal termination while a package is uninstalled In-Reply-To: <20110330174557.GA23664@mobee> References: <20110123130350.5392.80843.reportbug@mobee> <20110330154627.GE18485@yellowpig> <20110330174557.GA23664@mobee> Message-ID: <20110330204659.GC27249@yellowpig> On Wed, Mar 30, 2011 at 07:45:57PM +0200, Serafeim Zanikolas wrote: > Hi Bill, > > On Wed, Mar 30, 2011 at 05:46:27PM +0200, Bill Allombert wrote [edited]: > > As far as I understand, this is a warning, not an anormal termination. > > Yes, we could check the return value of open, but it this instance I > > feel that the warning is legitimate. > > Agreed, but the user that sees the warning cannot tell that. > > > For example, if the user set up a cron job that automatically update their system, > > it can alert them that there is a conflict. > > > > Please note that this bug was also reported as #555790. I am still not quite sure > > how to address it. > > Perhaps add a msg to make it clear that it is normal under certain > circumstances (along the lines of "failed to open xxx (this is normal if > dpkg was running at the same time)". This will save you from having to deal > with such bug reports in the future. Well, I have already changed the warning to read: "popcon: file $dpkg_db/$pkg.list is missing\n" Cheers, -- Bill. Imagine a large red swirl here. From sez at debian.org Wed Mar 30 20:59:45 2011 From: sez at debian.org (Serafeim Zanikolas) Date: Wed, 30 Mar 2011 22:59:45 +0200 Subject: [Popcon-developers] Bug#610860: popularity-contest: abnormal termination while a package is uninstalled In-Reply-To: <20110330204659.GC27249@yellowpig> References: <20110123130350.5392.80843.reportbug@mobee> <20110330154627.GE18485@yellowpig> <20110330174557.GA23664@mobee> <20110330204659.GC27249@yellowpig> Message-ID: <20110330205945.GB23664@mobee> On Wed, Mar 30, 2011 at 10:46:59PM +0200, Bill Allombert wrote: [..] > Well, I have already changed the warning to read: > "popcon: file $dpkg_db/$pkg.list is missing\n" Fine by me, cheers. -- Every great idea is worthless without someone to do the work. --Neil Williams From Bill.Allombert at math.u-bordeaux1.fr Thu Mar 31 12:47:22 2011 From: Bill.Allombert at math.u-bordeaux1.fr (Bill Allombert) Date: Thu, 31 Mar 2011 14:47:22 +0200 Subject: [Popcon-developers] Release 1.52 Message-ID: <20110331124722.GA1451@yellowpig> Hello Petter, I would like to release popcon 1.52 soon. I plan to do more changes, but I need to implement the server-side first. This version include a new debian/cron.daily that detect when popcon is not run for more than 7 days. I would appreciate if you are able to test it. Cheers, -- Bill. Imagine a large red swirl here. From owner at bugs.debian.org Thu Mar 31 12:54:14 2011 From: owner at bugs.debian.org (Debian Bug Tracking System) Date: Thu, 31 Mar 2011 12:54:14 +0000 Subject: [Popcon-developers] Bug#604449: marked as done ([popularity-contest] updated level2 d-i Greek translation) References: <20110331123221.GC1195@yellowpig> <4CEA482D.2060003@tee.gr> Message-ID: Your message dated Thu, 31 Mar 2011 14:32:21 +0200 with message-id <20110331123221.GC1195 at yellowpig> and subject line Re: Bug#604449: [popularity-contest] updated level2 d-i Greek translation has caused the Debian Bug report #604449, regarding [popularity-contest] updated level2 d-i Greek translation to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact owner at bugs.debian.org immediately.) -- 604449: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=604449 Debian Bug Tracking System Contact owner at bugs.debian.org with problems -------------- next part -------------- An embedded message was scrubbed... From: Emmanuel Galatoulas Subject: [popularity-contest] updated level2 d-i Greek translation Date: Mon, 22 Nov 2010 12:38:37 +0200 Size: 7143 URL: -------------- next part -------------- An embedded message was scrubbed... From: Bill Allombert Subject: Re: Bug#604449: [popularity-contest] updated level2 d-i Greek translation Date: Thu, 31 Mar 2011 14:32:21 +0200 Size: 2946 URL: From pere at hungry.com Thu Mar 31 13:01:34 2011 From: pere at hungry.com (Petter Reinholdtsen) Date: Thu, 31 Mar 2011 15:01:34 +0200 Subject: [Popcon-developers] Release 1.52 In-Reply-To: <20110331124722.GA1451@yellowpig> References: <20110331124722.GA1451@yellowpig> Message-ID: <20110331130134.GN29416@login2.uio.no> [Bill Allombert] > Hello Petter, Hi. > I would like to release popcon 1.52 soon. Good. Please go ahead and do not wait for me. > I plan to do more changes, but I need to implement the server-side > first. This version include a new debian/cron.daily that detect > when popcon is not run for more than 7 days. I would appreciate if > you are able to test it. I'm still busy handling my kinds, so I am unlikely to find time to test it. I hope I get my evenings back soon. :) Happy hacking, -- Petter Reinholdtsen