[Pkg-mailman-hackers] Bug#919158: mailman3-web: cronjobs should be run with flock to avoid them running amok

[Sampo Sorsa]

Package: mailman3-web
Version: 0+20180916-2~bpo9+1
Severity: serious

Dear Maintainer,

mailman3-web cronjobs (django-admin runjobs minutely as worst) are run without flock. When the system is (temporarily) constrained on resources, this can cause a severe backlog of django-admin runjobs, grinding the system to a halt.

This is not fun:

root       961  0.0  0.0  76116   104 ?        S    10:44   0:00     /usr/sbin/CRON -f
www-data   962  0.0  0.0   4288     0 ?        Ss   10:44   0:00       /bin/sh -c [ -f /usr/bin/django-admin ] && python3 /usr/bin/django-admin runjobs minutely --pythonpath /usr/share/mailman3-web --settings settings
www-data   963  0.3  0.0 343884  1808 ?        Sl   10:44   0:12         python3 /usr/bin/django-admin runjobs minutely --pythonpath /usr/share/mailman3-web --settings settings
root      1330  0.0  0.0  76116   108 ?        S    10:45   0:00     /usr/sbin/CRON -f
www-data  1335  0.0  0.0   4288     0 ?        Ss   10:45   0:00       /bin/sh -c [ -f /usr/bin/django-admin ] && python3 /usr/bin/django-admin runjobs minutely --pythonpath /usr/share/mailman3-web --settings settings
www-data  1337  0.2  0.2 339696 10836 ?        Sl   10:45   0:08         python3 /usr/bin/django-admin runjobs minutely --pythonpath /usr/share/mailman3-web --settings settings

Ad infinitum

# pidof -x python3
13186 12884 12592 12304 11987 11596 11154 11103 10890 10889 10888 10887 10886 10857 10607 10260 10206 10068 9671 9371 9021 8551 7579 7096 6646 6315 5983 5662 5351 5029 4705 4394 3974 3643 2976 2405 2357 2041 2036 1680 1337 963

# uptime
 11:48:37 up 29 days,  3:46, 21 users,  load average: 110.60, 99.61, 73.86

# kill -9 $(pidof -x python3)
-bash: fork: Cannot allocate memory

My PTSD now recommends prefixing each job in /etc/cron.d/mailman3-web with a flock -n call:

* *     * * *   www-data        [ -f /usr/bin/django-admin ] && flock -n /var/run/mailman3-web/cron.minutely python3 /usr/bin/django-admin runjobs minutely --pythonpath /usr/share/mailman3-web --settings settings
2,17,32,47 * * * * www-data     [ -f /usr/bin/django-admin ] && flock -n /var/run/mailman3-web/cron.quarter_hourly python3 /usr/bin/django-admin runjobs quarter_hourly --pythonpath /usr/share/mailman3-web --settings settings
@hourly         www-data        [ -f /usr/bin/django-admin ] && flock -n /var/run/mailman3-web/cron.hourly python3 /usr/bin/django-admin runjobs hourly   --pythonpath /usr/share/mailman3-web --settings settings
@daily          www-data        [ -f /usr/bin/django-admin ] && flock -n /var/run/mailman3-web/cron.daily python3 /usr/bin/django-admin runjobs daily    --pythonpath /usr/share/mailman3-web --settings settings
@weekly         www-data        [ -f /usr/bin/django-admin ] && flock -n /var/run/mailman3-web/cron.weekly python3 /usr/bin/django-admin runjobs weekly   --pythonpath /usr/share/mailman3-web --settings settings
@monthly        www-data        [ -f /usr/bin/django-admin ] && flock -n /var/run/mailman3-web/cron.monthly python3 /usr/bin/django-admin runjobs monthly  --pythonpath /usr/share/mailman3-web --settings settings
@yearly         www-data        [ -f /usr/bin/django-admin ] && flock -n /var/run/mailman3-web/cron.yearly python3 /usr/bin/django-admin runjobs yearly   --pythonpath /usr/share/mailman3-web --settings settings

Let me know if you want a patch.

By the way, is there a reason why /usr/share/mailman3-web/manage.py is not used throughout the mailman3-web package instead of "/usr/bin/django-admin --pythonpath /usr/share/mailman3-web --settings settings"?

Because that's basically what the manage.py wrapper does.

--
Sampo Sorsa