[Python-modules-team] Bug#958764: closed by Scott Kitterman <debian at kitterman.com> (re: python3-pip: debundled _vendor packaging approach breaks usage of pip in environments)

[Paul Ivanov]

Thanks for looking into this, Scott!

> Maintaining symlink farms is inherently difficult and unreliable.  Debian used 
> to use a similar approach to support multiple python2 versions and it never 
> worked well.  I would be reluctant to use such an approach.

Ok, makes sense.

> I checked with a pip upstream developer on IRC and his view is that what pipx 
> is doing is not supported by upstream:
>
> > <ScottK> dstufft: If an program external to pip tries to use one of pip's
> > vendored libraries directsly (e.g. from pip._vendor.six import iteritems)
> > this won't work on Debian because we don't put the wheels in the pip
> > package tree (they are in /usr/share/python-wheels).  From your point of
> > view upstream, if another package is doing such a thing are they
> > unreasonably mucking about in pip internals and if it breaks it's their
> > problem or is it something reasonable that they should be able to rely on
> > and we should try to support?
> > <dstufft> ScottK: upstrem does not support people importing pip internals
> > /vendored libs 
> > <dstufft> we moved *everything* into a _ directory just to make this obvious

Pipx is not importing pip internals. Pipx is simply using pip.
The way pipx is doing this is by adding the site-packages of a
working pip installation in it's ~/.local/pipx/shared folder to
the sys.path of the environments that it manages in various
~/.local/pipx/venvs folders

Pipx is trying to save from having a copy of pip in each of its
venvs. This works if the pip on the path is the  wheel as
published on PyPI, or as the vendored code with the debundled
wheels in the  pip/_vendor directory, with the debundled wheels
in the pip/_vendor directory. But this does not work with the
logic patched in Debian's packaging that points the debundled
wheels to the sys.prefix + "/share/python-wheels" , because the
sys.prefix for each of the venvs is different and will not have
copies of the vendored code.

So the question to follow up with Donald and PyPA folks is
something like: "should pip be usable by adding the site packages
of a working installation to the sys.path of another
environment?" And similarly, "should a repackaged pip wheel work
standalone" - because the pip wheel in /usr/share/python-wheels
cannot be used the same way as a regular pip wheel, it is not
standalone, and it requires making the debundled wheels which are
assumed to be on the sys.prefix + share/python-wheels

> It may be that this pipx bug is only apparent on Debian and its derivatives 
> because we rely on the pip unbundle logic, but what pipx is doing is 
> unsupported upstream, so I don't think we can support it either.

Again, I believe it's the additional modification to the unbundle
logic, namely that the vendored wheels will be in sys.prefix +
share/python-wheels directory *instead* of the pip/_vendor
directory that is causing this behaviour.

> Sorry I didn't have a more positive answer.

I appreciate that, thank you for your time and efforts.

best,
pi