[Python-modules-team] Bug#919929: python-scipy: autopkgtest fails (Re: bug#919929)

[Drew Parsons]

On 2019-03-10 15:46, Drew Parsons wrote:
> On 2019-03-10 03:51, Paul Gevers wrote:
>> Hi Drew,
>> 
> 
>>> To remove the deprecation warnings we'd need to deal with them at the
>>> source. Upstream has patches
>>> https://github.com/scipy/scipy/commit/614847c5fc8d5f8a618980df3c1b93540428ae46
>>> 
>>> https://github.com/scipy/scipy/commit/e0cfa29e2fbe86f994924c0c7514ff5bbfffd514
>>> 
>>> and for completeness
>>> https://github.com/scipy/scipy/commit/87e48c3c54d7a85bc6628c88c1de98ac0469b6fa
...
>>> Can you authorise an unblock to apply these 3 upstream patches to
>>> python-scipy 1.1.0 ?
>>> That won't necessarily fix the debci failure, but it will make it a 
>>> lot
>>> easier to see what's actually failing.
>> 
>> I am slightly unhappy with the second patch, as it seems to be doing
>> more than you describe above in a few places. This may be correct but
>> that is difficult to quickly judge.

I've adapted the 3 patches and pushed to salsa,
    matrix_API_614847c5.patch
    matrix_API_more_e0cfa29e2.patch
    matrix_API_filter_check_87e48c3c5.patch
https://salsa.debian.org/python-team/modules/python-scipy/tree/master/debian/patches

The other behaviour that you saw in the second patch I think might be 
replacement of "D*diag(v)" with "D at diag(v)". That's matrix 
multiplication, so still relevent to the matrix API patch. @ is not 
available in python2 so I changed it to numpy.matmul, which does the 
same thing.

The numpy.sparse tests pass with this patch, and most of the matrix 
PendingDeprecationWarnings are gone (the upstream patch missed 
integrate/tests/test_ivp.py, but the remaining warnings are few enough 
to not need to worry about).


>> Also, what is the general documented way that these wrappers can be
>> used? scipy is sort of taking over the maintenanceship of these
>> functions in this way if we're not careful.
> 
...
> There is discussion of the distinction between numpy.matrix and
> numpy.ndarray (which is at the heart of the deeprecation warning) at
> https://docs.scipy.org/doc/scipy/reference/tutorial/linalg.html#numpy-matrix-vs-2d-numpy-ndarray
> 
> The utility class scipy.sparse.sputils itself is apparently
> undocumented, by which I infer it's intended for internal use only,
> not a public API. I guess it's reasonable for a package to be testing
> it's own internal functions.  Strange thing is,
> scipy.sparse.sputils.matrix is not actually defined in
> scipy/sparse/sputils.py. Must be inherited or defined in some deep
> pythonfu that I haven't mastered yet.


Actually scipy.sparse.sputils.matrix was defined in these patches.  It 
is a bit odd that upstream is wrapping numpy.matrix just to avoid 
deprecation warnings, rather than following their own advice and using 
numpy.ndarray instead.


With these patches, the sparse matrix tests pass. There remain three 
errors unrelated to sparse matrix:
   spatial/tests/test__plotutils.py::TestPlotting::test_delaunay FAILED   
   [ 76%]
   spatial/tests/test__plotutils.py::TestPlotting::test_voronoi FAILED    
   [ 76%]
   spatial/tests/test__plotutils.py::TestPlotting::test_convex_hull 
FAILED  [ 76%]
   __________________________ TestPlotting.test_delaunay 
__________________________

   self = <scipy.spatial.tests.test__plotutils.TestPlotting object at 
0x7f0f31156eb8>

       def test_delaunay(self):
           # Smoke test
           fig = plt.figure()
           obj = Delaunay(self.points)
           s_before = obj.simplices.copy()
   >       with suppress_warnings as sup:
   E       AttributeError: __enter__

   fig        = <Figure size 640x480 with 0 Axes>
   obj        = <scipy.spatial.qhull.Delaunay object at 0x7f0f31163400>
   s_before   = array([[3, 1, 0],
          [2, 3, 0]], dtype=int32)
   self       = <scipy.spatial.tests.test__plotutils.TestPlotting object 
at 0x7f0f31156eb8>

   
/usr/lib/python3/dist-packages/scipy/spatial/tests/test__plotutils.py:31: 
AttributeError

(likewise for the other 2)
These AttributeErrors are mentioned at 
https://github.com/scipy/scipy/issues/9491. Upstream doesn't seem too 
concerned about them.

Apart from that, I'm happy to upload the sparse matrix patches once the 
s390x update reaches testing.

Drew