[Pkg-mailman-hackers] Bug#1014037: mailman3-web: Possible memory leak: uwsgi OOMs after a few weeks

[Antoine Beaupré]

Control: tags -1 -moreinfo

On 2024-04-25 01:59:07, Pierre-Elliott Bécue wrote:
> Peter Chubb <peter.chubb at unsw.edu.au> wrote on 29/06/2022 at 03:11:15+0200:

[...]

> Having the same kind of setup for the past 6 years, I never had such an
> issue.
>
> Do you have more intel?

What do you need? :)

We've been running Mailman 3 from Debian packages for a couple of months
now, and we're seeing recurring OOM errors. At first, we were hitting
8GB memory usage, and bumped the memory of that machine to 16GB, but
we're still getting OOMs. Our incident log is in:

https://gitlab.torproject.org/tpo/tpa/team/-/issues/41957

Here's a screenshot of a Grafana dashboard of our "per-process memory
exporter" that shows, well, per-process memory usage:

-------------- next part --------------
A non-text attachment was scrubbed...
Name: Screenshot 2025-01-15 at 09-58-38 Per-process memory usage - system - Dashboards - Grafana.png
Type: image/png
Size: 1531364 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/pkg-mailman-hackers/attachments/20250115/0a77f0dc/attachment-0001.png>
-------------- next part --------------

Here you can see uwsgi suddenly spiking up to 12GB of RAM, mostly
resident (RSS) memory.

It seems memory leaks is a known issue with uwsgi. Our investigation
shows there are numerous upstream bug reports about it, this search
counts 9 reports mentioning "memory leak":

https://github.com/unbit/uwsgi/issues?q=is%3Aissue+is%3Aopen+memory+leak

I should also mention that uwsgi is "in maintenance mode" according to
the upstream README.

This thread on the upstream mailman mailing list mentions people don't
have this kind of problem with gunicorn:

https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/QCTB7Y6W7I7GDRCIJKFNEVQB7DSNC4WG/

So I'm tempted to reassign this bug report to the uwsgi package,
although perhaps the mailman3-web package should work on switching to
the different process manager than uwsgi, at least as an option, to
resolve that issue.

We'll be experimenting with gunicorn and report back here.

a.

-- 
Being cynical is the only way to deal with modern civilization — you
can't just swallow it whole.
                        - Frank Zappa