[Python-modules-team] Bug#961206: improve sphinx usage for cross building

[Dmitry Shachnev]

Hi Helmut!

On Fri, Jun 12, 2020 at 01:02:26PM +0200, Helmut Grohne wrote:
> Hi Dmitry,
>
> > So what are our next steps? I will make a Sphinx 3.x upload with updated
> > package description to experimental soon.
>
> Unfortunately, I encountered another little roadblock. At least
> sphinx-apidoc imports modules. Doing so certainly isn't M-A:foreign. I
> don't know whether sphinx-autogen also does. In any case, this doesn't
> work as planned.

Ah, yes. One use of Sphinx is automatically generating documentation from
Python code, and it definitely requires importing code.

sphinx.ext.autodoc is one place where it happens, but some other extensions
(like sphinx.ext.coverage and sphinx.ext.inheritance_diagram) also do it.
Grepping the code for import_module gives an approximation of where else
this is used.

At least autodoc is very popular:
https://codesearch.debian.net/search?q=sphinx.ext.autodoc

I just thought that maybe this code could be moved to a separate package
(not M-A: foreign), but that would require a (permanent) circular dependency,
as these extensions use code from Sphinx core.

> Given that I've performed the analysis under the assumption that marking
> sphinx M-A:foreign was ok, the rest of this email is going to assume
> that even though this assumption is now known to be wrong.
>
> We could do with a temporary, circular dependency. We could split the
> sphinx package from python3-sphinx right now. Now sphinx must depend on
> python3-sphinx forever and python3-sphinx must depend on sphinx
> temporarily to avoid breaking rdeps. The latter dependency can be
> dropped in like half a year maybe.
>
> As soon as this split is in place, we can mark sphinx Multi-Arch:
> foreign (assuming this was fine). Once that happens, cross builds can
> immediately benefit from the split without requiring that everyone else
> updates their Build-Depends.

So the tricky part here is “assuming this was fine”. If that can be assumed,
then having a temporary circular dependency is OK for me.

> > Do you want to prepare mass bug filing, or you prefer that we deal with the
> > packages in Python team (DPMT/PAPT) first?
>
> I strongly prefer reducing the number of bugs to file before filing
> them.

I agree.

> There are also two parts of this transition:
>
>  A) Fixing packages to work without the python3-sphinx -> sphinx
>     dependency.
>  B) Fixingg packages to use sphinx instead of python3-sphinx for cross
>     building.
>
> I think B is a long-tail transition and you basically don't have to care
> about it at all. A is the transition you need to pay attention to.
>
> So how about numbers?
> 
> Around 1200 packages (transivitively) build depend on python3-sphinx. I
> built them with a path-exclude for the sphinx tools.  Around 500
> packages FTBFS when /usr/bin/sphinx-* is removed. Around 450 clearly
> fail at using one of the tools. The rest is roughly 25 filed FTBFS and
> 25 packages where I have no clue.
>
> Here is the list of packages that definitely use a sphinx commandline
> tool:
> [...]

So you already have a list of packages that need bugs, so if we decide to
proceed you can just file them in a bulk, asking to add sphinx
build-dependency (and possibly remove python3-sphinx).

> I'm quite unsure about how to proceed from here. The split doesn't work
> as planned. :-/

Same for me…

--
Dmitry Shachnev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/python-modules-team/attachments/20200613/7ab118e0/attachment.sig>