[Python-modules-team] Bug#961206: improve sphinx usage for cross building

[Helmut Grohne]

Package: python3-sphinx
Version: 2.4.3-2
Severity: wishlist
User: debian-cross at lists.debian.org
Usertags: cross-satisfiability

sphinx is becoming ever more popular and these days it is often used for
generating manual pages. That makes it required for building arch:any
packages (as opposed to arch:all *-doc) packages more often and this
affects cross building as a dependency on python3-sphinx is generally
not cross satisfiable. We cannot easily mark python3-sphinx Multi-Arch:
foreign either, so this leaves us in a difficult place.

This bug asks for improving the situation. It comes without a patch as I
seek consensus on how to make this work.

Generally, the rough idea would be marking python3-sphinx Multi-Arch:
foreign. Unfortunately, that is out of question, because the interface
of the python3-sphinx package inludes the sphinx Python module, which
happens to depend on markupsafe via jinja2. Since markupsafe is
architecture-dependent, there is no way to make python3-sphinx
Multi-Arch: foreign.

So the only option for improving the situation is making the interface
"smaller". In essence, this means providing the sphinx module and the
sphinx command line tools via different package names. A very rough
sketch for this would be moving sphinx-build and the other tools to a
new binary package maybe called simply "sphinx". Of course "sphinx"
would depend on python3-sphinx, but given that sphinx would only cover
the command line interface, sphinx could be marked Multi-Arch: foreign.
Does this make sense in principle when one ignores the transition cost?

Now getting there is not trivial. Simply moving sphinx-build an friends
to a new binary package is going to make a lot of packages FTBFS.
Obviously, that's not what we want. So the transition would roughly work
like this:
 1. python3-sphinx starts providing sphinx.
 2. We ask downstream packages to switch their python3-sphinx dependency
    to sphinx iff they call it via sphinx-build.  This affects at most
    1200 source packages. A lot of them are python modules and could be
    quickly fixed in git by some DPMT member. The actual affected
    packages could be determined with a partial archive rebuild.
 3. After a while and getting that number of 1200 down a little, we
    could start a MBF at severity normal.
 4. Once the number of broken rdeps is down to around 100, we could do
    the actual split and bump the remaining bugs to rc severity.

I doubt that this would finish in time for bullseye.

This does not solve cross building for advanced sphinx users where
sphinx extensions are involved. However that is not the majority of
sphinx users.

Since moving alternatives from one binary package to another is
difficult, I wonder whether we can rid of it now. Doing so would greatly
easy the projected steps.

Now the questions are:
 * Is the requested sphinx (the cli) and python3-sphinx (the module)
   split a reasonable thing to do?
 * Is the transition plan a reasonable thing to do?
 * Is this transition worth the cost (cross building vs changing lots of
   packages)?
 * Can we get rid of the use of update-alternatives?

Helmut