[Debian-med-packaging] Any chance to only partly disable tests for camitk (Re: r16503 - in trunk/packages/camitk/trunk/debian: . patches)

Fri Mar 28 11:36:04 UTC 2014

Hello Andreas and other debian-med members,

There are three main subjects in this email. Sorry for the length...

1) package test suite

the good news: I checked with the CamiTK developer team this afternoon 
and we managed to list the relevant test id (basically the one that pass 
out of the box, i.e., the one for which we already did the necessary 
code changes).

In order to select only some of the tests and as we are using ctest, I 
needed to execute the ctest command from the build directory.
My problem: I have no idea how to get the build directory name during 
the debian/rules execution (although browsing the web to find a 
solution, I came around other nice tricky bits, e.g., to get the current 
package version from the changelog!).

The problem then is to load the main library (libcamitkcore) during the 
test execution: this means to update the LD_LIBRARY_PATH to the current 
build directory.
And here I was stuck for a while:
a) I did not find anywhere how to get the default build directory name 
(on my test platform it is automatically set to "obj-x86_64-linux-gnu"
b) I did not find anyway of updating the LD_LIBRARY_PATH for the 
makefile subshell where I did the test: the default build directory is 
not created before the build target is run, therefore, setting 
LD_LIBRARY_PATH at the beginning of the debian/rules to include 
obj-*/lib and before the "export LD_LIBRARY_PATH" seems impossible.

The solution I found was to change the dh default build directory (using 
dh $@ --builddirectory=camitk-build)

Q: Is there any drawback doing this?
Q: Anyone knows how to get the default build directory name at the 
beginning of the makefile?
Q: Or to update the LD_LIBRARY_PATH before running the test / calling a 
subshell in a target?

Then I was able to run the test, but I ran into another problem: 
although all the selected test passed when run on sid, some of them 
failed when run in the pbuilder. I will investigate this when the test 
suite is fully operational.
So I decided to remove the incriminated tests...

We (upstream) will (try to) improve the test suite for the next release 
so that all tests are relevants.

2) autopkgtest

I also explore a little bit the autopkgtest solution, which looks great 
for automatically checking the installed package.

In this recent post from the autopkgtest maintainers [1], it is advised 
to first read [2], which gives you all the information you need.
Another good, but not complete, documentationto can be found in [3].

I added the first test in debian/tests. It is the test I usually do 
locally before committing on the debian-med svn (test the installation 
configuration using camitk-config --config as I explained in my previous 
post).

I was able to check my script in different ways:
1. using adt-run to check the current version available (in my case, it 
was on sid) :
sudo adt-run --no-built-binaries --built-tree=. --- adt-virt-null
(as explained in [2])
In this case the available package camitk 3.2.2 package is checked.

2. using adt-run to check the build version (in my case the new one)

These first two ways are probably the laziest way (no virtual environment).
I also tested adt-run using lxc [4]. Once I had a lxc VM (called 
camitk-sid in my case) up and ready, I was able to do for instance using 
the package produced by pbuilder:

sudo adt-run --no-built-binaries *.deb *.dsc --gnupg-home=fresh --- 
adt-virt-lxc camitk-sid

And here is the results:
...
adt-run: & dsc8t-config:  - - - - - - - - - - results - - - - - -
dsc8t-config         PASS
adt-run: & dsc8t-config:  - - - - - - - - - - stdout - - - - - -
Detected CamiTK version is 3.3.0
Checking configuration...
CamiTK 3.3.0 configuration: OK
adt-run: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ tests done.

This is really great!

If I understand [1] correctly this means that once uploaded, the camitk 
package will be checked on http://ci.debian.net/

I will add more test when time allows (checking the compilation of 
camitk extension).

For both testing (testsuite and autopkgtest) I had to add xvfb and xauth 
to the dependencies, they are needed by our tests (the config and test 
applications both need an X11 context).

3) CamiTK 3.3.0 ready(?)

Would this be ok now to upload the new version? I checked the last 
committed changes on sid and in a pbuilder environment + the adt-run

Only two lintian warnings are left:
W: camitk source: unknown-field-in-dsc testsuite
W: camitk source: newer-standards-version 3.9.5 (current is 3.9.4)

The first one is due to the new field in control for autpkg:
XS-Testsuite: autopkgtest
Which I suppose is ok (?) considering I added the X-TestSuite flag in 
debian/control in order to activate autopkgtest

For this second one, I let Andreas decide.

Thanks again for helping us improve this package and encouraging me to 
explore autopkgtest.

Best regards,
Mahnu

[1] https://lists.debian.org/debian-devel-announce/2014/02/msg00010.html
[2] 
http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=blob_plain;f=doc/README.running-tests
[3] http://packaging.ubuntu.com/html/auto-pkg-test.html
[4] https://wiki.debian.org/LXC

On 24/03/14 11:38, Andreas Tille wrote:
> Hi Emmanuel,
>
> On Mon, Mar 24, 2014 at 10:48:03AM +0100, Emmanuel Promayon wrote:
>>
>> Q: can I run the tests independently after a dpkg-buildpackage?
>> And/or what is the best way to run the tests?
>
> As Charles said: sadt (in devtools package) might be your friend but I
> have (not yet) tested it myself.
>
> Kind regards
>
>         Andreas.
>

-- 
Emmanuel Promayon
UJF-Grenoble 1, CNRS, TIMC-IMAG UMR 5525 (équipe GMCAO)
Institut de l'Ingénierie de l'Information de Santé
Faculté de Médecine - 38706 La Tronche cedex - France
Tel. +33/0 456 52 00 03 - Fax. +33/0 456 52 00 55 - B7

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2947 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.alioth.debian.org/pipermail/debian-med-packaging/attachments/20140328/52c58eb3/attachment.bin>