Bug#898526: h5py: FTBFS with HDF5 1.10.2
Gilles Filippini
pini at debian.org
Thu May 17 18:05:18 BST 2018
Control: tags + patch
On Sun, 13 May 2018 16:01:31 +0200 Gilles Filippini <pini at debian.org> wrote:
> Control: severity -1 serious
> Control: retitle -1 h5py: FTBFS - FAIL: test_out_of_order_offsets
>
> On Sun, 13 May 2018 02:49:12 +0200 Gilles Filippini <pini at debian.org> wrote:
> > Source: h5py
> > Version: 2.7.1-2
> > Severity: normal
> >
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA512
> >
> > Hi,
> >
> > h5py FTBFS with HDF5 1.10.2 currently in experimental. The failure occurs
> > during the tests where two of them fail:
> >
> > ======================================================================
> > FAIL: test_out_of_order_offsets (h5py.tests.hl.test_datatype.TestOffsets)
> > - ----------------------------------------------------------------------
> > Traceback (most recent call last):
> > File "h5py/tests/hl/test_datatype.py", line 198, in test_out_of_order_offsets
> > self.assertArrayEqual(fd['data'], data)
> > File "h5py/tests/common.py", line 124, in assertArrayEqual
> > "Dtype mismatch (%s vs %s)%s" % (dset.dtype, arr.dtype, message)
> > AssertionError: Dtype mismatch ({'names':['f1','f3','f2'], 'formats':['<f4','<f8','<i4'], 'offsets':[0,8,16], 'itemsize':20} vs {'names':['f1','f2','f3'], 'formats':['<f4','<i4','<f8'], 'offsets':[0,16,8], 'itemsize':20})
> >
> > ======================================================================
> > FAIL: test_out_of_order_offsets (h5py.tests.old.test_h5t.TestCompound)
> > - ----------------------------------------------------------------------
> > Traceback (most recent call last):
> > File "h5py/tests/old/test_h5t.py", line 61, in test_out_of_order_offsets
> > self.assertEqual(tid.dtype, expected_dtype)
> > AssertionError: dtype({'names':['f1','f3','f2'], 'formats':['<f4','<f8','<i4'], 'offsets':[0,8,16], 'itemsize':20}) != dtype({'names':['f1','f2','f3'], 'formats':['<f4','<i4','<f8'], 'offsets':[0,16,8], 'itemsize':20})
> >
> > - ----------------------------------------------------------------------
> > Ran 447 tests in 1.206s
> >
> > FAILED (failures=2, skipped=18, expected failures=6)
>
> Actually it FTBFS with the very same failure on unstable as well. Then
> raising severity to serious.
>
> This seems tied to the recent python-numpy upgrade to 1.14.3. It builds
> fine against python-numpy 1.13.3.
Upstream release 2.8.0 doesn't FTBFS. The upstream commit 5009e06 [1]
fixes the issue. Attached a patch proposal after this commit.
Thanks,
-------------- next part --------------
diff -Nru h5py-2.7.1/debian/changelog h5py-2.7.1/debian/changelog
--- h5py-2.7.1/debian/changelog 2017-09-11 11:15:49.000000000 +0200
+++ h5py-2.7.1/debian/changelog 2018-05-16 17:00:35.000000000 +0200
@@ -1,3 +1,10 @@
+h5py (2.7.1-2.1) UNRELEASED; urgency=medium
+
+ * Non-maintainer upload.
+ * Backport upstream commit to support numpy 1.14
+
+ -- Gilles Filippini <pini at debian.org> Wed, 16 May 2018 17:00:35 +0200
+
h5py (2.7.1-2) unstable; urgency=medium
* Fixup the debug package description for Python 2.
diff -Nru h5py-2.7.1/debian/patches/series h5py-2.7.1/debian/patches/series
--- h5py-2.7.1/debian/patches/series 2017-09-11 11:15:49.000000000 +0200
+++ h5py-2.7.1/debian/patches/series 2018-05-16 17:00:35.000000000 +0200
@@ -1,3 +1,4 @@
No-rpath.patch
No-intersphinx.patch
Fix-build-of-API-docs-with-Python-3.patch
+Support-numpy-1.14.patch
diff -Nru h5py-2.7.1/debian/patches/Support-numpy-1.14.patch h5py-2.7.1/debian/patches/Support-numpy-1.14.patch
--- h5py-2.7.1/debian/patches/Support-numpy-1.14.patch 1970-01-01 01:00:00.000000000 +0100
+++ h5py-2.7.1/debian/patches/Support-numpy-1.14.patch 2018-05-16 17:00:35.000000000 +0200
@@ -0,0 +1,84 @@
+Description: Backport of upstream commit 5009e06 stripped of setup related
+ changes
+
+From 5009e062a6f7d4e074cab0fcb42a780ac2b1d7d4 Mon Sep 17 00:00:00 2001
+From: James Tocknell <aragilar at gmail.com>
+Date: Thu, 28 Dec 2017 20:55:55 +1100
+Subject: [PATCH] FIX: Don't reorder compound types, breaks on numpy 1.14
+
+---
+ h5py/h5t.pyx | 25 +++++++------------------
+ setup.py | 2 +-
+ tox.ini | 4 ++--
+ 3 files changed, 10 insertions(+), 21 deletions(-)
+
+diff --git a/h5py/h5t.pyx b/h5py/h5t.pyx
+index cc2344e1..7445e9eb 100644
+--- a/h5py/h5t.pyx
++++ b/h5py/h5t.pyx
+@@ -1136,12 +1136,6 @@ cdef class TypeCompoundID(TypeCompositeID):
+ else:
+ if sys.version[0] == '3':
+ field_names = [x.decode('utf8') for x in field_names]
+- if len(field_names) > 0:
+- collated_fields = zip(field_names, field_types, field_offsets)
+- ordered_fields = sorted(
+- collated_fields, key=operator.itemgetter(2))
+- field_names, field_types, field_offsets = \
+- map(list, zip(*ordered_fields))
+ typeobj = dtype({
+ 'names': field_names,
+ 'formats': field_types,
+@@ -1458,8 +1452,7 @@ cdef TypeCompoundID _c_compound(dtype dt, int logical, int aligned):
+ cdef dtype member_dt
+ cdef size_t member_offset = 0
+
+- cdef dict offsets = {}
+- cdef list fields = []
++ cdef dict fields = {}
+
+ # The challenge with correctly converting a numpy/h5py dtype to a HDF5 type
+ # which is composed of subtypes has three aspects we must consider
+@@ -1468,19 +1461,14 @@ cdef TypeCompoundID _c_compound(dtype dt, int logical, int aligned):
+ # 2. For correct round-tripping of aligned dtypes, we need to consider how
+ # much padding we need by looking at the field offsets
+ # 3. There is no requirement that the offsets be monotonically increasing
+- # (so we start by sorting the names as a function of increasing offset)
+ #
+ # The code below tries to cover these aspects
+
+- # Get offsets for each compound member
+- for name, field in dt.fields.items():
+- offsets[name] = field[1]
+-
+ # Build list of names, offsets, and types, sorted by increasing offset
+ # (i.e. the position of the member in the struct)
+- for name in sorted(dt.names, key=offsets.__getitem__):
++ for name in sorted(dt.names, key=(lambda n: dt.fields[n][1])):
+ field = dt.fields[name]
+- name = name.encode('utf8') if isinstance(name, unicode) else name
++ h5_name = name.encode('utf8') if isinstance(name, unicode) else name
+
+ # Get HDF5 data types and set the offset for each member
+ member_dt = field[0]
+@@ -1489,7 +1477,7 @@ cdef TypeCompoundID _c_compound(dtype dt, int logical, int aligned):
+ if aligned and (member_offset > field[1]
+ or member_dt.itemsize != member_type.get_size()):
+ raise TypeError("Enforced alignment not compatible with HDF5 type")
+- fields.append((name, member_offset, member_type))
++ fields[name] = (h5_name, member_offset, member_type)
+
+ # Update member offset based on the HDF5 type size
+ member_offset += member_type.get_size()
+@@ -1500,8 +1488,9 @@ cdef TypeCompoundID _c_compound(dtype dt, int logical, int aligned):
+
+ # Create compound with the necessary size, and insert its members
+ tid = H5Tcreate(H5T_COMPOUND, member_offset)
+- for (name, member_offset, member_type) in fields:
+- H5Tinsert(tid, name, member_offset, member_type.id)
++ for name in dt.names:
++ h5_name, member_offset, member_type = fields[name]
++ H5Tinsert(tid, h5_name, member_offset, member_type.id)
+
+ return TypeCompoundID(tid)
+
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 508 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/debian-science-maintainers/attachments/20180517/964db9c6/attachment.sig>
More information about the debian-science-maintainers
mailing list