[Debian-med-packaging] Bug#1044060: q2-quality-control: test failure with pandas 2.0

[s3v]

Hi,

According to [1]:

    Empty DataFrames/Series will now default to have a ``RangeIndex``
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

    Before, constructing an empty (where ``data`` is ``None`` or an empty list-like argument) :class:`Series` or :class:`DataFrame` without
    specifying the axes (``index=None``, ``columns=None``) would return the axes as empty :class:`Index` with object dtype.

    Now, the axes return an empty :class:`RangeIndex` (:issue:`49572`).

    *Previous behavior*:

    .. code-block:: ipython

    In [8]: pd.Series().index
    Out[8]:
    Index([], dtype='object')

    In [9] pd.DataFrame().axes
    Out[9]:
    [Index([], dtype='object'), Index([], dtype='object')]

    *New behavior*:

    .. ipython:: python

    pd.Series().index
    pd.DataFrame().axes

empty dataframes are not equals anymore with pandas > 2.0 in this case:

test.py
-------

import pandas as pd
from pandas.testing import assert_frame_equal

df1 = pd.DataFrame(columns=['s1', 's2', 's3']).astype(float)  
df2 = pd.DataFrame(columns=['s1', 's2', 's3'], index=[]).astype(float)  

try:
   assert_frame_equal(df1, df2)
except AssertionError:
   raise

print('OK')


Output with pandas 2.1
----------------------
> python3.11 test.py  
Traceback (most recent call last):
 File "/build/test.py", line 8, in <module>
   assert_frame_equal(df1,df2)
 File "/usr/lib/python3/dist-packages/pandas/_testing/asserters.py", line 1156, in assert_frame_equal
   assert_index_equal(
 File "/usr/lib/python3/dist-packages/pandas/_testing/asserters.py", line 260, in assert_index_equal
   _check_types(left, right, obj=obj)
 File "/usr/lib/python3/dist-packages/pandas/_testing/asserters.py", line 236, in _check_types
   assert_attr_equal("inferred_type", left, right, obj=obj)
 File "/usr/lib/python3/dist-packages/pandas/_testing/asserters.py", line 414, in assert_attr_equal
   raise_assert_detail(obj, msg, left_attr, right_attr)
 File "/usr/lib/python3/dist-packages/pandas/_testing/asserters.py", line 598, in raise_assert_detail
   raise AssertionError(msg)
AssertionError: DataFrame.index are different

Attribute "inferred_type" are different
[left]:  integer
[right]: empty


Output with pandas 1.5
----------------------
> python3.11 test.py
OK

Please note the attached patch makes autopkgtestpass as well with
pandas 1.5.2.


On Sun, 18 Feb 2024 14:45:29 +0100 Andreas Tille <andreas at an3as.eu> wrote:
> Hi again,
>
> Am Sun, Feb 18, 2024 at 12:25:49PM +0100 schrieb Andreas Tille:
> > I just realised that a new qiime version is out. I will upgrade
> > to latest upstream and see how this might affect this issue
>
> The new qiime upstream version does not change anything. After I
> switched q2-* packages to run autopkgtest for `py3versions -s` I
> realised the problem below exist in several of the q2-* packages so its
> rather no Pandas issue but a Python3.12 problem which in parallel to the
> Pandas migration showed up.
>
> If you might have any hint how to deal with these (no matter for the
> old or the new qiime package since I assume the patch will apply
> to both, it would be really appreciated.
>
> Kind regards
> Andreas.
>
> > Am Sun, Feb 18, 2024 at 12:11:04PM +0100 schrieb Andreas Tille:
> > > Control: tags -1 help
> > >
> > > Hi again,
> > >
> > > I hope to approach the last remaining Pandas issue for the qiime
> > > ecosystem. As it has become obvious in the q2-types package I'm now
> > > facing pretty similar errors when running the q2-quality-control
> > > package which can be seen in full length in Salsa-CI[3] and contains
> > > errors like:
> > >
> > > E AttributeError: 'ProvenancePath' object has no attribute '_drv'
> > > E AttributeError: 'ProvenancePath' object has no attribute '_raw_paths'
> > > E AttributeError: 'ProvenancePath' object has no attribute '_str'
> > > E AttributeError: 'OutPath' object has no attribute '_str'
> > >
> > > This all goes back to the qiime package but I admit I have no idea
> > > how to fix this.
> > >
> > > Kind regards
> > > Andreas.
> > >
> > >
> > > [3] https://salsa.debian.org/med-team/q2-quality-control/-/jobs/5320775#L700
> > >
> > > Am Sat, Feb 17, 2024 at 11:36:41AM +0100 schrieb Andreas Tille:
> > > > Hi,
> > > >
> > > > as reported in a qiime2 issue[1] there is some problem with Python3.12
> > > > in the tests of the q2-* packages which are all using the qiime package.
> > > > This problem is currently hidden from the tests made by Python3.12
> > > > porters but it became obvious now on Salsa CI[2]. I tried to fiddle
> > > > around a bit with the qiime code but with no success at all. Any help
> > > > would be welcome.
> > > >
> > > > Kind regards
> > > > Andreas.
> > > >
> > > > [1] https://github.com/qiime2/qiime2/issues/751
> > > > [2] https://salsa.debian.org/med-team/q2-types/-/jobs/5313640#L900
> > > >

I really don't know how to fix this issue with python 3.12, sorry :(
 
Kind Regards

[1] https://sources.debian.org/src/pandas/2.1.4%2Bdfsg-4/doc/source/whatsnew/v2.0.0.rst/#L586










-------------- next part --------------
A non-text attachment was scrubbed...
Name: pandas2.0.patch
Type: text/x-patch
Size: 507 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/debian-med-packaging/attachments/20240219/e9ad0773/attachment-0001.bin>