[Python-apps-team] Bug#918953: itstool: Messed up with encoding when output is redirected (regression?)

[Boyuan Yang]

Package: itstool
Version: 2.0.5-2
Severity: important
X-Debbugs-CC: tanguy+debian at ortolo.eu tanguy at debian.org osamu at debian.org
Control: affects -1 debmake-doc

Dear itstool maintainer,

We recently found a potential regression that broke the building process for
package debmake-doc. A minimal example that can reproduce the problem is
attached with this report in a tarball.

% tree minimal_reproduce 
minimal_reproduce
├── debmake-doc.en.x02
├── docbook.its
├── run_me_crash
└── run_me_no_crash

0 directories, 4 files

When you run the "run_me_crash" script, itstool will return with error:

-> % cat run_me_crash 
#!/bin/sh
itstool -i ./docbook.its ./debmake-doc.en.x02 -o - | cat

-> % cat run_me_no_crash 
#!/bin/sh
itstool -i ./docbook.its ./debmake-doc.en.x02 -o -

% ./run_me_crash > /dev/null
Traceback (most recent call last):
  File "/usr/bin/itstool", line 1578, in <module>
    messages.output(out)
  File "/usr/bin/itstool", line 155, in output
    out.write(msg.format())
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in position
70: ordinal not in range(128)

-> % ./run_me_no_crash
(will run successfully)

-> % ./run_me_no_crash > /dev/null
Traceback (most recent call last):
  File "/usr/bin/itstool", line 1578, in <module>
    messages.output(out)
  File "/usr/bin/itstool", line 155, in output
    out.write(msg.format())
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in position
70: ordinal not in range(128)

===============================================================

Well that is really strange. As long as there's any redirection of output,
itstool will use 'ascii' encoding by default, which is weird and it's causing
problems. I guess it's a bug in itstool itself.

Tanguy, could you please take a look?

--
Regards,
Boyuan Yang
-------------- next part --------------
A non-text attachment was scrubbed...
Name: itstool_crash.scripts.tar.gz
Type: application/x-compressed-tar
Size: 83066 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/python-apps-team/attachments/20190110/0ac5e29e/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 858 bytes
Desc: This is a digitally signed message part
URL: <http://alioth-lists.debian.net/pipermail/python-apps-team/attachments/20190110/0ac5e29e/attachment-0001.sig>