[Pkg-mailman-hackers] Bug#785642: Bug#785642: queue runner dies with uncaught UnicodeDecodeError

[Thijs Kinkhorst]

On Mon, May 18, 2015 19:18, Wouter Verhelst wrote:
> Package: mailman
> Version: 1:2.1.18-2
> Severity: grave
> Justification: causes data loss
>
> Hi,
>
> I received a message from one of my list admins that he couldn't send a
> mail to his list. Investigating turned up the following in
> /var/log/mailman/error:
>
> May 17 15:32:20 2015 (981) Uncaught runner exception: 'utf8' codec can't
> decode byte 0xe9 in position 18: invalid continuation byte
> May 17 15:32:20 2015 (981) Traceback (most recent call last):
>   File "/var/lib/mailman/Mailman/Queue/Runner.py", line 119, in _oneloop
>     self._onefile(msg, msgdata)
>   File "/var/lib/mailman/Mailman/Queue/Runner.py", line 190, in _onefile
>     keepqueued = self._dispose(mlist, msg, msgdata)
>   File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 130, in
> _dispose
>     more = self._dopipeline(mlist, msg, msgdata, pipeline)
>   File "/var/lib/mailman/Mailman/Queue/IncomingRunner.py", line 153, in
> _dopipeline
>     sys.modules[modname].process(mlist, msg, msgdata)
>   File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 239, in
> process
>     i18ndesc = uheader(mlist, mlist.description, 'List-Id',
> maxlinelen=998)
>   File "/var/lib/mailman/Mailman/Handlers/CookHeaders.py", line 65, in
> uheader
>     return Header(s, charset, maxlinelen, header_name, continuation_ws)
>   File "/usr/lib/python2.7/email/header.py", line 183, in __init__
>     self.append(s, charset, errors)
>   File "/usr/lib/python2.7/email/header.py", line 267, in append
>     ustr = unicode(s, incodec, errors)
> UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 18:
> invalid continuation byte
>
> May 17 15:32:20 2015 (981) SHUNTING:
> 1431869540.389822+156779307d54473d0eb732994bb67eee95733285
>
> I'm not sure why mailman needs to decode the data in the mail as part of

I agree that this may be too strict. Can you provide the mail that caused
this?

> running the queue, but at any rate that shouldn't case the mail to get
> lost...

It's not lost, it's shunted, so I'm inclined to downgrade the bug as there
is no dataloss I think.

Cheers,
Thijs