Bug#743184: perl: SpamAssassin spamd segfaults since updating servers to Debian wheezy

[Niko Tyni]

On Mon, Mar 31, 2014 at 11:21:24AM +0200, Martin von Wittich wrote:
> Package: perl
> Version: 5.14.2-21+deb7u1
> Severity: normal

> since we've updated our customer servers (about 1000) to Debian wheezy,
> we're seeing a lot of "spamd child" segfaults on many of these servers.
> I've already filed a bug to the spamassassin package[1] back in Dec 2013
> when we first noticed the issue, but after investigating the issue, I
> think I've now isolated the cause in the Debian packaging of Perl
> itself, so I'm filing this bug now against the Perl package.

> In the last seven days, spamd segfaulted on 64 of our customer servers (out of
> a total of 932 servers that I checked). I have attached three backtraces to the
> original spamd bug report; please see those for details.

Thanks for the report and the work you've put into investigating this. 

The bug seems unfortunately hard to reproduce by others, so I'm going
to have to ask you to help some more. See below. If you can come up with
a recipe to trigger a crash, that would help a lot.

> To isolate the issue, I have run the following experiments on one of the
> affected servers (SA = SpamAssassin):
> 
> 2014-02-13   I installed SA 3.4.0 manually with cpanm[2].
> 
> 2014-02-25   spamd has crashed 10 times since 2014-02-14,
>              so a manual SA installation didn't resolve it.
> 2014-02-25   I compiled Perl 5.18.2 with Perlbrew[3] and again
>              installed SA 3.4.0 with cpanm.
> 
> 2014-03-10   spamd has crashed 0 times since 2014-02-26, so the custom
>              Perl 5.18.2 build doesn't seem to be affected.
> 2014-03-10   I compiled Perl 5.14.2 with Perlbrew and SA 3.4.0
>              with cpanm to make sure that the issue isn't caused by
>              a Perl bug that may have been fixed in Perl 5.18.2.
> 
> 2014-03-28   spamd has crashed 0 times since 2014-03-11,
>              so the custom Perl 5.14.2 build isn't affected either.
> 
> This leads me to my conclusion that the issue must be somehow caused by the
> Debian packaging of Perl... maybe a Debian patch or something?

While that is a possibility, I can't think of a non-upstream patch we're
carrying that could cause such segfaults.

However, there are a few backported patches both from 5.14.3 and 5.16/5.18
that might be relevant (mainly fixes/encode-memleak and fixes/regexp-*).
Our 5.14.2 in wheezy is really pretty much 5.14.3 without the version
increment (due to release/freeze scheduling), and important changes from
5.14.4 should be there too.

One thing to check is the configuration of the perlbrew-built perl binary.
The issue could be triggered only with the specific configuration options
that we're using in Debian. I'm mostly thinking of -Dusethreads and
-Duse64bitint here, and (to a lesser extent) static vs. dynamic linking
against libperl.

So, as a further data point, could you please try with upstream 5.14.4,
compiled with at least -Dusethreads -Duse64bitint -Duselargefiles, and
maybe also the gcc flags from dpkg-buildflags ? (See debian/config.debian
in the perl source package.) Note that these compilation options change
the ABI so you'll have to reinstall all the CPAN modules too.
-- 
Niko Tyni   ntyni at debian.org