[Python-apps-team] Bug#880172: alot: UnicodeEncodeError for search queries with non-ascii characters

[Johannes Schauer]

Package: alot
Version: 0.6-1
Severity: normal
Tags: patch

Hi,

steps to reproduce:

 1. search for a query string containing non-ascii characters

 2. try opening any of the resulting threads

 3. the following will be output in the debug log:

ERROR:ui:Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 651, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/share/alot/alot/ui.py", line 679, in call_apply
    return defer.maybeDeferred(cmd.apply, self)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 150, in maybeDeferred
    result = f(*args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1445, in unwindGenerator
    return _inlineCallbacks(None, gen, Deferred())
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1299, in _inlineCallbacks
    result = g.send(result)
  File "/usr/share/alot/alot/commands/globals.py", line 77, in apply
    ui.cleanup()
  File "/usr/share/alot/alot/ui.py", line 711, in cleanup
    self._cmd_hist_file, size=size)
  File "/usr/share/alot/alot/ui.py", line 764, in _save_history_to_file
    histfile.write(line)
exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 11: ordinal not in range(128)


alot will not crash but the result of this exception is, that all
messages in the thread are unfolded and not only those matching the
query terms. Naturally this is undesirable. To fix this problem, the
following patch to alot/db/message.py suffices:

@@ -254,5 +254,5 @@
 
     def matches(self, querystring):
         """tests if this messages is in the resultset for `querystring`"""
-        searchfor = '( {} ) AND id:{}'.format(querystring, self._id)
+        searchfor = u'( {} ) AND id:{}'.format(querystring, self._id)
         return self._dbman.count_messages(searchfor) > 0



Please consider applying it.

Thanks!

cheers, josch



-- System Information:
Debian Release: 9.0
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable-debug'), (500, 'unstable'), (500, 'oldstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386, armhf

Kernel: Linux 4.9.0-3-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages alot depends on:
ii  python               2.7.13-2
ii  python-configobj     5.0.6-2
ii  python-gpg           1.8.0-3+b2
ii  python-gpgme         0.3-1.2
ii  python-magic         1:5.30-1
ii  python-notmuch       0.23.7-3
ii  python-twisted       16.6.0-2
ii  python-twisted-core  16.6.0-2
ii  python-urwid         1.3.1-2+b1
ii  python-urwidtrees    1.0.1.1-1

Versions of packages alot recommends:
ii  links    2.14-2+b1
ii  links2   2.14-2+b1
ii  notmuch  0.23.7-3
ii  w3m      0.5.3-34

Versions of packages alot suggests:
pn  alot-doc  <none>

-- no debconf information

-- debsums errors found:
debsums: changed file /usr/share/alot/alot/db/message.py (from alot package)