[Pkg-mailman-hackers] Bug#785642: queue runner dies with uncaught UnicodeDecodeError

[Wouter Verhelst]

Package: mailman
Version: 1:2.1.18-2
Severity: grave
Justification: causes data loss

Hi,

I received a message from one of my list admins that he couldn't send a
mail to his list. Investigating turned up the following in
/var/log/mailman/error:

May 17 15:32:20 2015 (981) Uncaught runner exception: 'utf8' codec can't decode byte 0xe9 in position 18: invalid continuation byte
May 17 15:32:20 2015 (981) Traceback (most recent call last):
  File "/var/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop
    self._onefile(msg, msgdata)
  File "/var/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
    keepqueued = self._dispose(mlist, msg, msgdata)
  File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in _dispose
    more = self._dopipeline(mlist, msg, msgdata, pipeline)
  File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in _dopipeline
    sys.modules[modname].process(mlist, msg, msgdata)
  File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 239, in process
    i18ndesc = uheader(mlist, mlist.description, 'List-Id', maxlinelen=998)
  File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 65, in uheader
    return Header(s, charset, maxlinelen, header_name, continuation_ws)
  File "/usr/lib/python2.7/email/header.py", line 183, in __init__
    self.append(s, charset, errors)
  File "/usr/lib/python2.7/email/header.py", line 267, in append
    ustr = unicode(s, incodec, errors)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 18: invalid continuation byte

May 17 15:32:20 2015 (981) SHUNTING: 1431869540.389822+156779307d54473d0eb732994bb67eee95733285

I'm not sure why mailman needs to decode the data in the mail as part of
running the queue, but at any rate that shouldn't case the mail to get
lost...

-- System Information:
Debian Release: 8.0
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: armel (armv5tel)

Kernel: Linux 3.16.0-4-kirkwood
Locale: LANG=nl_BE.UTF-8, LC_CTYPE=nl_BE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages mailman depends on:
ii  apache2 [httpd]              2.4.10-10
ii  apache2-mpm-prefork [httpd]  2.4.10-10
ii  cron                         3.0pl1-127
ii  debconf [debconf-2.0]        1.5.56
ii  libc6                        2.19-18
ii  logrotate                    3.8.7-1+b1
ii  lsb-base                     4.1+Debian13+nmu1
ii  python-dnspython             1.12.0-1
pn  python:any                   <none>
ii  ucf                          3.0030

Versions of packages mailman recommends:
ii  exim4-daemon-heavy [mail-transport-agent]  4.84-8

Versions of packages mailman suggests:
pn  listadmin     <none>
pn  lynx          <none>
ii  spamassassin  3.4.0-6

-- debconf information:
* mailman/create_site_list:
* mailman/used_languages: en fr nl
* mailman/gate_news: false
* mailman/default_server_language: fr
  mailman/queue_files_present: abort installation
* mailman/site_languages: fr, en, nl