[Python-modules-team] Bug#907370: TypeError: object of type 'int' has no len()

[Peter Pentchev]

Control: tags -1 + upstream patch

On Sun, Aug 26, 2018 at 11:51:32PM -0400, Antoine Beaupre wrote:
> Package: python3-sh
> Version: 1.11-1
> Severity: grave
> 
> I found a regression in the python3-sh package. I have yet to find
> exactly what is going on, but it looks to me like something changed in
> Python 3.6 that broke the sh module in some way:
> 
> $ lwn get -o blog/2017-12-13-kubecon-overview.mdwn https://lwn.net/Articles/741301/
> Exception in thread Thread-1:
> Traceback (most recent call last):
>   File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
>     self.run()
>   File "/usr/lib/python3.6/threading.py", line 864, in run
>     self._target(*self._args, **self._kwargs)
>   File "/usr/lib/python3/dist-packages/sh.py", line 1453, in input_thread
>     done = stdin.write()
>   File "/usr/lib/python3/dist-packages/sh.py", line 1799, in write
>     self.log.debug("got chunk size %d: %r", len(proc_chunk),
> TypeError: object of type 'int' has no len()
> 
> The source for the `lwn` command is here:
> 
> https://gitlab.com/anarcat/lwn/blob/master/lwn.py
> 
> It's totally unclear to me, from the backtrace, from *where* exactly
> sh is being called, so I am having a hard time writing a
> reproducer... But I am *guessing* it's this call:
> 
>     result = sh.pandoc(filter_args, _in=content)
> 
> But really, I have no frigging clue.
> 
> Any brilliant idea of what could possibly be going on here? This used
> to work fine in Debian stretch (which has the same version of
> python3-sh, so probably no change there).
> 
> My best guess so far is something changed in some standard library,
> but I really have no idea...

I'm not really sure what changed to cause this bug, but I can certainly
reproduce it; moreover, it has been reported as a problem upstream[1]
and then fixed[2].  The attached patch (line numbers adapted for 1.11)
fixes it for me; an upgrade of python-sh to 1.12 or later would also fix
it.

Thanks for reporting this!

Best regards,
Peter

[1] https://github.com/amoffat/sh/issues/325
[2] https://github.com/amoffat/sh/commit/1460f6ee7ab521e1c443acdff8ec9f44069a196b

-- 
Peter Pentchev  roam@{ringlet.net,debian.org,FreeBSD.org} pp at storpool.com
PGP key:        http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint 2EE7 A7A5 17FC 124C F115  C354 651E EFB0 2527 DF13
-------------- next part --------------
Description: Allow Python 3 bytes as command input.
 commit 1460f6ee7ab521e1c443acdff8ec9f44069a196b
 Author: Andrew Moffat <andrew.robert.moffat at gmail.com>
 Date:   Wed Oct 5 22:34:53 2016 -0500
 .
     bugfix where python3 woudln't accept binary stdin
 .   
     closes #325
Debian-Bug: https://bugs.debian.org/907370
Origin: upstream; https://github.com/amoffat/sh/commit/1460f6ee7ab521e1c443acdff8ec9f44069a196b
Author: Andrew Moffat <andrew.robert.moffat at gmail.com>
Last-Update: 2018-09-04

--- a/sh.py
+++ b/sh.py
@@ -1652,6 +1652,10 @@
         log_msg = "string"
         get_chunk = get_iter_string_reader(input_obj)
 
+    elif isinstance(input_obj, bytes):
+        log_msg = "bytes"
+        get_chunk = get_iter_string_reader(input_obj)
+
     else:
         log_msg = "general iterable"
         get_chunk = get_iter_chunk_reader(iter(input_obj))
--- a/test.py
+++ b/test.py
@@ -614,6 +614,16 @@
         self.assertTrue(out == "")
 
 
+    def test_binary_input(self):
+        py = create_tmp_test("""
+import sys
+data = sys.stdin.read()
+sys.stdout.write(data)
+""")
+        data = b'1234'
+        out = python(py.name, _in=data)
+        self.assertEqual(out, "1234")
+
 
     def test_err_to_out(self):
         py = create_tmp_test("""
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/python-modules-team/attachments/20180904/e7818c03/attachment-0001.sig>