[Pkg-mailman-hackers] Bug#878597: mailman: messages shunted due to uncaught runner exception (UnicodeDecodeError in pipermail.py)

[Ben Elliston]

Package: mailman
Version: 1:2.1.23-1+deb9u1
Severity: important

Dear Maintainer,

At some point (not necessarily the most recent Mailman update), list traffic
has been shunted due to the following exception:

Oct 14 10:56:46 2017 (5640) SHUNTING: 1507939004.126139+5eab5884866a8622adc0e9588dadbc0faca8e4f4
Oct 14 11:02:24 2017 (5640) Uncaught runner exception: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte
Oct 14 11:02:24 2017 (5640) Traceback (most recent call last):
  File "/var/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop
    self._onefile(msg, msgdata)
  File "/var/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)

  [traceback trimmed for brevity]

  File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 640, in add_article
    author = fixAuthor(article.decoded['author'])
  File "/var/lib/mailman/Mailman/Archiver/pipermail.py", line 63, in fixAuthor
    while i>0 and (L[i-1][0] in lowercase or
UnicodeDecodeError: 'utf8' codec can't decode byte 0xaa in position 26: invalid start byte


The outcome I am expecting is that these messages not be shunted in
the first place.  I have search high and low for a similar bug report.
There have been similar reports relating to header munging, but nothing
involving list archiving.

None of my lists use non-standard encodings and they are all set to 'en'.

-- System Information:
Debian Release: 9.2
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: i386 (i686)

Kernel: Linux 4.9.0-3-686-pae (SMP w/1 CPU core)
Locale: LANG=en_AU, LC_CTYPE=en_AU (charmap=ISO-8859-1), LANGUAGE=en_AU:en_US:en_GB:en (charmap=ISO-8859-1)
Shell: /bin/sh linked to /bin/bash
Init: systemd (via /run/systemd/system)

Versions of packages mailman depends on:
ii  apache2 [httpd]        2.4.25-3+deb9u3
ii  cron [cron-daemon]     3.0pl1-128+b1
ii  debconf [debconf-2.0]  1.5.61
ii  libc6                  2.24-11+deb9u1
ii  logrotate              3.11.0-0.1
ii  lsb-base               9.20161125
ii  python                 2.7.13-2
ii  python-dnspython       1.15.0-1
ii  ucf                    3.0036

Versions of packages mailman recommends:
ii  postfix [mail-transport-agent]  3.1.6-0+deb9u1

Versions of packages mailman suggests:
ii  listadmin     2.42-1
pn  lynx          <none>
ii  spamassassin  3.4.1-6

-- debconf information:
  mailman/gate_news: false
* mailman/queue_files_present: continue regardless
* mailman/used_languages: en
* mailman/create_site_list:
* mailman/site_languages: en
* mailman/default_server_language: en