[Pkg-mailman-hackers] Bug#1014037: mailman3-web: Possible memory leak: uwsgi OOMs after a few weeks

[Michael Tremer]

Hello,

> On 5 Mar 2025, at 21:32, Antoine Beaupré <anarcat at debian.org> wrote:
> 
> On 2025-02-24 11:51:16, Antoine Beaupré wrote:
>> On 2025-02-24 15:56:57, Michael Tremer wrote:
>> 
>> [...]
>> 
>>> So, has this solved it all for good for you guys? What release of xapian are you on?
>> 
>> In the end, no, not really. Things have *improved*: we used to have 4-5
>> OOM/day, with peaks at 15, 120 when reindexing, and this is down to 1-5
>> a day, depending on the day. Kind of hard to track discrete events like
>> this...
>> 
>> We had a single OOM in the last 48h. That's "nice".
> 
> I have, at this point, closed the issue on our end:
> 
> https://gitlab.torproject.org/tpo/tpa/team/-/issues/41957
> 
> We've gone from 20-40 OOMs/week (multiple daily) to ~3 per week, so
> Xapian has definitely improved the situation.
> 
> I don't think this bug report should be closed though: we still have a
> memory leak issue. I don't think it's reasonable for mailman to take
> 16GB of RAM for such small setups. Xapian is also using an unreasonable
> amount of disk space.

I have been looking at alternatives to mailman3 recently. I think that there is a very good chance we would migrate to mlmmj. There are currently too many large outstanding problems with mailman and it seems that there is not enough of a community around it to get them fixed in time. Although the large memory consumption is mostly annoying and not a deal-breaker, mailman keeps stopping to accept emails sometimes and needs a restart.

However, mlmmj (also packaged in Debian) is super small and super simple. The feature set is exactly what we need, although it would have been nice to have an API to subscribe/unsubscribe users. Since it is all a collection of small binaries, that can be built very easily with a couple of CGI scripts or so.

But it does not have any archiving features beyond storing all emails in a directory. So there is public-inbox which has a simple web UI, stores emails in a Git repositories which can be cloned and backed up very easily, *and* it is using Xapian indexes. So I thought I would give this a go and import our lists into it - just so that I have a way to compare.

On mailman, my Xapian index is about 4.9 GiB, on public-inbox I have 1.4 GiB. A significant change. This is still kind of large, but roughly only a quarter. The search also seems to be much faster. So I assume there is some configuration here that makes the index a lot smaller and the smaller the index the faster the search usually.

The mlmmj + public-inbox solution seems to have gained a lot of traction recently. The Linux kernel people are using it (https://lore.kernel.org/), Gentoo is using it (https://public-inbox.gentoo.org/), Promox, the list is actually quote long. So I think we might have a better chance to get something back that worked as well as mailman 2 without all this large complexity.

> But for all intents and purposes, this is as much effort I can dedicate
> to this. Hopefully, when we upgrade to trixie, we can run a profiler on
> this.

I agree. This is not the most painful problem in the world, but our mailing list needs to *just work* and I cannot spend a lot of time on keeping it working.

I would still be curious to find out what the actual problem was here though...

> Feel free to remind me of that in a year.
> 
> Otherwise happy to provide more info as needed of course.
> 
> Cheers!
> 
> -- 
> Ce que les siècles des grands abatoirs nous aura appris
> Devrait être inscrit au fond de toutes les écoles;
> Voici l'homme: le destructeur des mondes est arrivé.
>                        - [no one is innocent]