[Pkg-puppet-devel] Bug#1078911: missing reports cleanup job

[Antoine Beaupre]

Package: puppetserver
Version: 7.9.5-2
Severity: grave

On our "good old" puppetmaster server running 5.5, we have a cron job
that cleans up old reports, part of the Debian package:

root at pauli:/etc# cat cron.daily/puppet-master 
#!/bin/sh

if [ -e /var/lib/puppet/reports ] ; then
	find /var/lib/puppet/reports -maxdepth 2 -type f -ctime +30 -delete
fi

exit 0
root at pauli:/etc# dpkg -S cron.daily/puppet-master
puppet-master: /etc/cron.daily/puppet-master

(In fact, we have *another* similar job here:

root at pauli:/etc# cat cron.d/local-puppet-reports
@daily root [ -d /var/lib/puppet/reports ] && find /var/lib/puppet/reports -type f -mtime +30 -delete && find /var/lib/puppet/reports -type d -empty -delete

... where that is from is a mystery to me... Also notice how it checks mtime
instead of ctime, doesn't have the empty cleanup, aand the maxdepth. Anyways.)

On my home lab running the shiny new puppetserver from bookworm (well,
not so new anymore, but anyway), I *don't* have such a job. And today,
/var ran out of disk space, with /var/lib/puppet/reports taking a
whopping 45GiB of disk space. 

I stopped the bleeding by purging all the 2023 reports by hand, which
freed up 10GiB, but it really seems to me we dropped something in that
upgrade there, we should have a cron job or systemd timer that does that
cleanup automatically.

What exactly the command should be doesn't seem to matter much, all
those three are equivalent on my end:

root at marcos:/home/anarcat# find /var/lib/puppet/reports -maxdepth 2 -type f -ctime +30 | wc -l 
5242
root at marcos:/home/anarcat# find /var/lib/puppet/reports -maxdepth 2 -type f -mtime +30 | wc -l 
5242
root at marcos:/home/anarcat# find /var/lib/puppet/reports -type f -mtime +30 | wc -l 
5242

And it finds only one empty directory, from a node that hasn't ran in
ages and should probably be considered lost/retired at this point. Still
seems like a good idea to add that as well.

I mark this as grave because it can "cause data loss" (in my case
bounced email) due to unbounded disk usage.

A.

-- System Information:
Debian Release: 12.6
  APT prefers stable-security
  APT policy: (500, 'stable-security'), (500, 'stable-debug'), (500, 'stable'), (1, 'unstable'), (1, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 6.1.0-23-amd64 (SMP w/12 CPU threads; PREEMPT)
Locale: LANG=fr_CA.UTF-8, LC_CTYPE=fr_CA.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages puppetserver depends on:
ii  default-jre-headless                         2:1.17-74
ii  facter                                       4.3.0-2
ii  hiera                                        3.10.0-1
ii  jruby                                        9.3.9.0+ds-8
ii  libclj-time-clojure                          0.15.2-2
ii  libclj-yaml-clojure                          0.7.2-1
ii  libclojure-java                              1.11.1-2
ii  libcomidi-clojure                            0.3.2-2
ii  libcommons-exec-java                         1.3-2
ii  libcommons-io-java                           2.11.0-2
ii  libcommons-lang-java                         2.6-10
ii  libdropwizard-metrics-java                   3.2.6-1
ii  libdujour-version-check-clojure              0.2.3-1
ii  libjruby-utils-clojure                       4.0.3-4
ii  libkitchensink-clojure                       3.2.1-1
ii  libliberator-clojure                         0.15.3-1
ii  libprismatic-schema-clojure                  1.2.0-4
ii  libpuppetlabs-http-client-clojure            2.1.1-1
ii  libpuppetlabs-i18n-clojure                   0.9.2-2
ii  libpuppetlabs-ring-middleware-clojure        1.3.1-3
ii  libraynes-fs-clojure                         1.5.2-1
ii  libsemver-clojure                            0.3.0-2
ii  libshell-utils-clojure                       1.0.2-3
ii  libslingshot-clojure                         0.12.2-3
ii  libssl-utils-clojure                         3.5.0-2
ii  libtrapperkeeper-authorization-clojure       1.0.0-4
ii  libtrapperkeeper-clojure                     3.2.0-4
ii  libtrapperkeeper-comidi-metrics-clojure      0.1.2-2
ii  libtrapperkeeper-filesystem-watcher-clojure  1.2.2-3
ii  libtrapperkeeper-metrics-clojure             1.5.0-5
ii  libtrapperkeeper-scheduler-clojure           1.1.3-7
ii  libtrapperkeeper-status-clojure              1.1.1-4
ii  libtrapperkeeper-webserver-jetty9-clojure    4.4.1-5
ii  libyaml-snake-java                           1.33-2
ii  puppet-agent                                 7.23.0-1
ii  ruby                                         1:3.1
ii  ruby-deep-merge                              1.1.1-2
ii  ruby-fast-gettext                            2.0.3-2
ii  ruby-gettext                                 3.3.3-2
ii  ruby-hocon                                   1.3.1-2
ii  ruby-locale                                  2.1.3-1
ii  ruby-puppet-resource-api                     1.8.16-2
ii  ruby-puppetserver-ca-cli                     2.4.0-4
ii  ruby-semantic-puppet                         1.0.4-1
ii  ruby-text                                    1.3.1-1

Versions of packages puppetserver recommends:
ii  puppet-module-puppetlabs-augeas-core   1.1.2-1
ii  puppet-module-puppetlabs-cron-core     1.1.0+dfsg1-1
pn  puppet-module-puppetlabs-host-core     <none>
pn  puppet-module-puppetlabs-mount-core    <none>
pn  puppet-module-puppetlabs-selinux-core  <none>
ii  puppet-module-puppetlabs-sshkeys-core  2.3.0-1

puppetserver suggests no packages.

-- Configuration Files:
/etc/puppet/puppetserver/conf.d/auth.conf [Errno 13] Permission non accordée: '/etc/puppet/puppetserver/conf.d/auth.conf'
/etc/puppet/puppetserver/conf.d/ca.conf [Errno 13] Permission non accordée: '/etc/puppet/puppetserver/conf.d/ca.conf'
/etc/puppet/puppetserver/conf.d/global.conf [Errno 13] Permission non accordée: '/etc/puppet/puppetserver/conf.d/global.conf'
/etc/puppet/puppetserver/conf.d/metrics.conf [Errno 13] Permission non accordée: '/etc/puppet/puppetserver/conf.d/metrics.conf'
/etc/puppet/puppetserver/conf.d/puppetserver.conf [Errno 13] Permission non accordée: '/etc/puppet/puppetserver/conf.d/puppetserver.conf'
/etc/puppet/puppetserver/conf.d/web-routes.conf [Errno 13] Permission non accordée: '/etc/puppet/puppetserver/conf.d/web-routes.conf'
/etc/puppet/puppetserver/conf.d/webserver.conf [Errno 13] Permission non accordée: '/etc/puppet/puppetserver/conf.d/webserver.conf'
/etc/puppet/puppetserver/logback.xml [Errno 13] Permission non accordée: '/etc/puppet/puppetserver/logback.xml'
/etc/puppet/puppetserver/request-logging.xml [Errno 13] Permission non accordée: '/etc/puppet/puppetserver/request-logging.xml'
/etc/puppet/puppetserver/services.d/bootstrap.cfg [Errno 13] Permission non accordée: '/etc/puppet/puppetserver/services.d/bootstrap.cfg'
/etc/puppet/puppetserver/services.d/ca.cfg [Errno 13] Permission non accordée: '/etc/puppet/puppetserver/services.d/ca.cfg'

-- no debconf information