[Python-modules-team] Bug#744145: pip3 breaks after upgrading requests

[Chris Kuehl]

Hi again maintainer,

As Yannick suggests, this seems to have been caused by this change to requests
back in July:
https://github.com/kennethreitz/requests/commit/47d0517d66e8cf5832768262221f0357ae134ad1

Now that requests.compat no longer imports IncompleteRead, it won't be
available for pip to use.

There are two sequential commits in pip which remove its use of
IncompleteRead:

https://github.com/pypa/pip/commit/53eab2357e4425436b51e109bfaedcb9f3772072
https://github.com/pypa/pip/commit/e77c0c573c059bb4a86b2c5c06835f57e4af6c47

These were merged as part of the PR: https://github.com/pypa/pip/pull/2024

Unfortunately, the commits themselves don't contain much context for the
change. It looks like it was perhaps made to satisfy CI warnings?

On Tue, Oct 21, 2014 at 10:14:49PM +0200, Yannick Roehlly wrote:
> A quick fix would be to use "from httplib import IncompleteRead" in the
> de-vendorize.patch (even if the IncompleteRead will not be raised by requests
> any more).

I've attached a patch that does this (with a minor change to support Python3),
and tested that it works.

I've also attached a patch which backports the two pip commits into the current
python-pip source package.

Both of these fix the current problem, and allow pip to work with both the
version of requests the Debian package, and the current version from PyPI,
which is likely to be installed by users. I'm not sure which is preferable, so
I've included both (only one should be applied, of course).

> Note that there may be other problems with pip using the upgraded requests.

This is probably also worth reiterating. I wonder how hard it would be to
change system pip to always use system versions of libraries. There's no
guarantee that a change to the internals of requests doesn't break pip again
tomorrow.

Thanks again!
Chris


On Fri, Jan 23, 2015 at 01:34:16AM -0800, Chris Kuehl wrote:
> Hi maintainer,
> 
> I am able to reproduce this reliably.
> 
> On a clean jessie or sid system (does not happen on wheezy):
> 
>     root# apt-get install python3 python3-pip
>     root# pip3 install --upgrade requests
> 
> After this, every invocation of pip3 fails:
> 
>     root# pip3
>     Traceback (most recent call last):
>       File "/usr/bin/pip3", line 9, in <module>
>         load_entry_point('pip==1.5.6', 'console_scripts', 'pip3')()
>       File "/usr/lib/python3/dist-packages/pkg_resources.py", line 356, in load_entry_point
>         return get_distribution(dist).load_entry_point(group, name)
>       File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2476, in load_entry_point
>         return ep.load()
>       File "/usr/lib/python3/dist-packages/pkg_resources.py", line 2190, in load
>         ['__name__'])
>       File "/usr/lib/python3/dist-packages/pip/__init__.py", line 61, in <module>
>         from pip.vcs import git, mercurial, subversion, bazaar  # noqa
>       File "/usr/lib/python3/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
>         from pip.download import path_to_url
>       File "/usr/lib/python3/dist-packages/pip/download.py", line 25, in <module>
>         from requests.compat import IncompleteRead
>     ImportError: cannot import name 'IncompleteRead'
> 
> The version of requests which the upgrade installs is currently 2.5.1.
> 
> This renders pip unusable for many cases, and can break not just pip but
> also other Python packages. In particular, any Python package that
> depends on requests which you upgrade using system pip will permanently
> break pip.
> 
> Thanks for looking into this -- I'm more than happy to help with testing
> any fixes!
> 
> Happy Friday,
> Chris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-backport-remove-incompleteread.patch
Type: text/x-diff
Size: 3350 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/python-modules-team/attachments/20150123/858a4d50/attachment.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-import-httplib.patch
Type: text/x-diff
Size: 1607 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/python-modules-team/attachments/20150123/858a4d50/attachment-0001.patch>