Bug#940679: pandas: random test crashes

Rebecca N. Palmer rebecca_palmer at zoho.com
Sun Sep 22 16:59:53 BST 2019


Control: tags -1 patch

Running the test suite (of -7) under gdb a few times produced a crash in 
pandas/tests/indexes/interval/test_astype.py::TestDatetimelikeSubtype::test_astype_category[index4] 
(4 not 3).

The trace points to Cython-generated C from _Timedelta._has_ns, which 
(among other things) tries to dereference 
((PyDictObject*)((_PyObject_GetDictPtr(((PyObject *)__pyx_v_self))))).

(gdb) p (__pyx_v_self)
$18 = (struct __pyx_obj_6pandas_5_libs_6tslibs_10timedeltas__Timedelta 
*) 0x7fff8ee09f70
(gdb) p *((PyDictObject*)((_PyObject_GetDictPtr(((PyObject 
*)__pyx_v_self)))))
Cannot access memory at address 0x7fff8ee0a000 # __pyx_v_self+144
(gdb) p ((PyDictObject*)((_PyObject_GetDictPtr(((PyObject 
*)__pyx_v_self)))))
$5 = (PyDictObject *) 0x7fff8ee09ff0 # __pyx_v_self+128
(gdb) p sizeof(PyDictObject)
$28 = 48 # if it starts at +128, it ends at +176

(gdb) p ((PyObject *)__pyx_v_self)->ob_type->tp_dictoffset
$25 = 128
(gdb) p ((PyObject *)__pyx_v_self)->ob_type->tp_weaklistoffset
$29 = 136
(gdb) p ((PyObject *)__pyx_v_self)->ob_type->tp_basicsize
$26 = 144
(gdb) p sizeof(*(__pyx_v_self))
$21 = 128 # PyDictObject is off the end of the C struct

This looks like Python thinks _Timedelta has a per-instance dict, but C 
thinks it doesn't and so doesn't allocate space for it, and the crash 
happens when Python tries to read off the end.

I suspect this is because cdef classes aren't supposed to have non-cdef 
attributes, which makes this an easy problem to fix:

--- a/pandas/_libs/tslibs/timedeltas.pyx
+++ b/pandas/_libs/tslibs/timedeltas.pyx
@@ -655,7 +655,7 @@ cdef class _Timedelta(timedelta):
          int64_t _d, _h, _m, _s, _ms, _us, _ns

      # higher than np.ndarray and np.matrix
-    __array_priority__ = 100
+    cdef public object __array_priority__ = 100

      def __hash__(_Timedelta self):
          if self._has_ns():

but the Cython documentation is ambiguous as to whether this is actually 
prohibited, and I haven't tested it yet.



More information about the debian-science-maintainers mailing list