Bug#870128: libgpuarray: broken autopkgtests

Steve Langasek steve.langasek at canonical.com
Tue Aug 29 06:37:05 UTC 2017 

Hi Ghislain,

On Sun, Jul 30, 2017 at 01:16:14PM +0100, Ghislain Vaillant wrote:

> Indeed, which allows running the upstream test suite on our CPU-based
> builders. This is better than the default autopkgtest-pkg-python, which
> does nothing but testing the import of the Python module.

> Your assessment is wrong. pocl is a **CPU** implementation of OpenCL,
> so the tests should run in principle. The issue is that the test suite
> does not skip tests for which a specific compute must be satisfied by
> the environment, such as CUDA and float16. The rest of the tests runs
> fine.

Ah, thanks for the clarification.

> The old basic does nothing. For such package, testing whether the
> import is successful is hardly testing at all.

Fair enough.

>> I am filing this bug because failing autopkgtests are considered blockers
>> for inclusion of a package in the Ubuntu release.  Newer versions of
>> libgpuarray might not be included in Ubuntu releases until this is
>> resolved.

> I believe this is wrong. How can we provide upstream with meaningful
> logs without enabling the tests in the first place? I can keep the old
> autopkgtest-pkg-python to satisfy Ubuntu's policy, but what good would
> it make?

Well, you can generate whatever logs you want, the issue is when
known-bad tests are treated as a *failure* of the autopkgtest.  This
prevents the autopkgtests from being used for continuous integration,
because there is no signal to say when the test results are good.

If the testing framework supports marking these tests as XFAIL as part of
the autopkgtest, or otherwise skipping them when run on unsuitable hardware,
I think that would be preferable.

Attached is a patch that appears to achieve this here.

Cheers,
-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek at ubuntu.com                                     vorlon at debian.org
-------------- next part --------------
diff -Nru libgpuarray-0.6.9/debian/changelog libgpuarray-0.6.9/debian/changelog
--- libgpuarray-0.6.9/debian/changelog	2017-08-05 09:29:35.000000000 -0700
+++ libgpuarray-0.6.9/debian/changelog	2017-08-28 23:30:28.000000000 -0700
@@ -1,3 +1,10 @@
+libgpuarray (0.6.9-2ubuntu1) artful; urgency=medium
+
+  * debian/patches/skip-cuda-tests: skip those tests that we know will
+    fail on opencl.  Closes: #870128.
+
+ -- Steve Langasek <steve.langasek at ubuntu.com>  Mon, 28 Aug 2017 23:30:28 -0700
+
 libgpuarray (0.6.9-2build1) artful; urgency=medium
 
   * No-change rebuild to build to drop python3.5.
diff -Nru libgpuarray-0.6.9/debian/patches/series libgpuarray-0.6.9/debian/patches/series
--- libgpuarray-0.6.9/debian/patches/series	1969-12-31 16:00:00.000000000 -0800
+++ libgpuarray-0.6.9/debian/patches/series	2017-08-28 23:19:34.000000000 -0700
@@ -0,0 +1 @@
+skip-cuda-tests
diff -Nru libgpuarray-0.6.9/debian/patches/skip-cuda-tests libgpuarray-0.6.9/debian/patches/skip-cuda-tests
--- libgpuarray-0.6.9/debian/patches/skip-cuda-tests	1969-12-31 16:00:00.000000000 -0800
+++ libgpuarray-0.6.9/debian/patches/skip-cuda-tests	2017-08-28 23:30:28.000000000 -0700
@@ -0,0 +1,90 @@
+Description: skip those tests that we know will fail on opencl
+ Some of the upstream tests require CUDA in order to run.  Skip these tests
+ instead of treating them as errors when called in a non-CUDA environment.
+Author: Steve Langasek <steve.langasek at ubuntu.com>
+Bug-Debian: https://bugs.debian.org/870128
+
+Index: libgpuarray-0.6.9/pygpu/tests/test_elemwise.py
+===================================================================
+--- libgpuarray-0.6.9.orig/pygpu/tests/test_elemwise.py
++++ libgpuarray-0.6.9/pygpu/tests/test_elemwise.py
+@@ -2,6 +2,7 @@
+ import numpy
+ from mako.template import Template
+ 
++from nose.plugins.skip import SkipTest
+ from unittest import TestCase
+ from pygpu import gpuarray, ndgpuarray as elemary
+ from pygpu.dtypes import dtype_to_ctype, get_common_dtype
+@@ -43,7 +44,10 @@
+     c, g = gen_gpuarray((50,), dtype, ctx=context, cls=elemary)
+ 
+     out_c = op(c)
+-    out_g = op(g)
++    try:
++        out_g = op(g)
++    except gpuarray.GpuArrayException:
++        raise SkipTest("skipping test on this hardware")
+ 
+     assert out_c.shape == out_g.shape
+     assert out_c.dtype == out_g.dtype
+@@ -123,7 +127,10 @@
+                           cls=elemary)
+ 
+     out_c = op(ac, bc)
+-    out_g = op(ag, bg)
++    try:
++        out_g = op(ag, bg)
++    except gpuarray.GpuArrayException:
++        raise SkipTest("skipping test on this hardware")
+ 
+     assert out_c.shape == out_g.shape
+     assert out_c.dtype == out_g.dtype
+@@ -147,7 +154,10 @@
+         # TODO: currently, we use old Numpy semantic and tolerate more case.
+         # So we can't test that we raise the same error
+         return
+-    out_g = op(ag, bg)
++    try:
++        out_g = op(ag, bg)
++    except gpuarray.GpuArrayException:
++        raise SkipTest("skipping test on this hardware")
+ 
+     assert out_g is ag
+     assert numpy.allclose(out_c, numpy.asarray(out_g), atol=1e-6)
+Index: libgpuarray-0.6.9/pygpu/tests/test_gpu_ndarray.py
+===================================================================
+--- libgpuarray-0.6.9.orig/pygpu/tests/test_gpu_ndarray.py
++++ libgpuarray-0.6.9/pygpu/tests/test_gpu_ndarray.py
+@@ -8,6 +8,7 @@
+ 
+ import numpy
+ 
++from nose.plugins.skip import SkipTest
+ from nose.tools import assert_raises
+ import pygpu
+ from pygpu.gpuarray import GpuArray, GpuKernel
+@@ -193,7 +194,10 @@
+ 
+     # numpy upcast with a view to 1d scalar.
+     if gpu.flags['F_CONTIGUOUS']:
+-        assert b.gpudata == gpu.gpudata
++        try:
++            assert b.gpudata == gpu.gpudata
++        except TypeError:
++            raise SkipTest("skipping test on this hardware")
+     elif (sliced != 1 or shp == () or (offseted_outer and len(shp) > 1) or
+           (order != 'f' and len(shp) > 1)):
+         assert b is not gpu
+@@ -284,7 +288,10 @@
+ def mapping_getitem_ellipsis(shp, dtype, offseted):
+     a, a_gpu = gen_gpuarray(shp, dtype, offseted, ctx=ctx)
+     b = a_gpu[...]
+-    assert b.gpudata == a_gpu.gpudata
++    try:
++        assert b.gpudata == gpu.gpudata
++    except TypeError:
++        raise SkipTest("skipping test on this hardware")
+     assert b.strides == a.strides
+     assert b.shape == a.shape
+     b_cpu = numpy.asarray(b)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/debian-science-maintainers/attachments/20170828/af056e9c/attachment.sig>

More information about the debian-science-maintainers mailing list