Bug#940679: pandas: random test crashes
Rebecca N. Palmer
rebecca_palmer at zoho.com
Sun Sep 22 16:59:53 BST 2019
Control: tags -1 patch
Running the test suite (of -7) under gdb a few times produced a crash in
pandas/tests/indexes/interval/test_astype.py::TestDatetimelikeSubtype::test_astype_category[index4]
(4 not 3).
The trace points to Cython-generated C from _Timedelta._has_ns, which
(among other things) tries to dereference
((PyDictObject*)((_PyObject_GetDictPtr(((PyObject *)__pyx_v_self))))).
(gdb) p (__pyx_v_self)
$18 = (struct __pyx_obj_6pandas_5_libs_6tslibs_10timedeltas__Timedelta
*) 0x7fff8ee09f70
(gdb) p *((PyDictObject*)((_PyObject_GetDictPtr(((PyObject
*)__pyx_v_self)))))
Cannot access memory at address 0x7fff8ee0a000 # __pyx_v_self+144
(gdb) p ((PyDictObject*)((_PyObject_GetDictPtr(((PyObject
*)__pyx_v_self)))))
$5 = (PyDictObject *) 0x7fff8ee09ff0 # __pyx_v_self+128
(gdb) p sizeof(PyDictObject)
$28 = 48 # if it starts at +128, it ends at +176
(gdb) p ((PyObject *)__pyx_v_self)->ob_type->tp_dictoffset
$25 = 128
(gdb) p ((PyObject *)__pyx_v_self)->ob_type->tp_weaklistoffset
$29 = 136
(gdb) p ((PyObject *)__pyx_v_self)->ob_type->tp_basicsize
$26 = 144
(gdb) p sizeof(*(__pyx_v_self))
$21 = 128 # PyDictObject is off the end of the C struct
This looks like Python thinks _Timedelta has a per-instance dict, but C
thinks it doesn't and so doesn't allocate space for it, and the crash
happens when Python tries to read off the end.
I suspect this is because cdef classes aren't supposed to have non-cdef
attributes, which makes this an easy problem to fix:
--- a/pandas/_libs/tslibs/timedeltas.pyx
+++ b/pandas/_libs/tslibs/timedeltas.pyx
@@ -655,7 +655,7 @@ cdef class _Timedelta(timedelta):
int64_t _d, _h, _m, _s, _ms, _us, _ns
# higher than np.ndarray and np.matrix
- __array_priority__ = 100
+ cdef public object __array_priority__ = 100
def __hash__(_Timedelta self):
if self._has_ns():
but the Cython documentation is ambiguous as to whether this is actually
prohibited, and I haven't tested it yet.
More information about the debian-science-maintainers
mailing list