[Debian-med-packaging] Any chance to only partly disable tests for camitk (Re: r16503 - in trunk/packages/camitk/trunk/debian: . patches)

[Andreas Tille]

Hi Emmanuel,

On Fri, Mar 28, 2014 at 12:36:04PM +0100, Emmanuel Promayon wrote:
> 
> 1) package test suite
> 
> the good news: I checked with the CamiTK developer team this
> afternoon and we managed to list the relevant test id (basically the
> one that pass out of the box, i.e., the one for which we already did
> the necessary code changes).
> 
> In order to select only some of the tests and as we are using ctest,
> I needed to execute the ctest command from the build directory.
> My problem: I have no idea how to get the build directory name
> during the debian/rules execution (although browsing the web to find
> a solution, I came around other nice tricky bits, e.g., to get the
> current package version from the changelog!).
> 
> The problem then is to load the main library (libcamitkcore) during
> the test execution: this means to update the LD_LIBRARY_PATH to the
> current build directory.
> And here I was stuck for a while:
> a) I did not find anywhere how to get the default build directory
> name (on my test platform it is automatically set to
> "obj-x86_64-linux-gnu"
> b) I did not find anyway of updating the LD_LIBRARY_PATH for the
> makefile subshell where I did the test: the default build directory
> is not created before the build target is run, therefore, setting
> LD_LIBRARY_PATH at the beginning of the debian/rules to include
> obj-*/lib and before the "export LD_LIBRARY_PATH" seems impossible.
> 
> The solution I found was to change the dh default build directory
> (using dh $@ --builddirectory=camitk-build)
> 
> Q: Is there any drawback doing this?

As far as I can see from just reading your mail (= without testing)
setting --builddirectory is totally normal and should not have any
unwanted side effects.

> Q: Anyone knows how to get the default build directory name at the
> beginning of the makefile?

I think setting --builddirectory is exactly the way to go.

> Q: Or to update the LD_LIBRARY_PATH before running the test /
> calling a subshell in a target?

I think this could work as well.  In general I would (strongly)
advise to direct this kind of questions to
   debian-mentors at lists.debian.org
since there are more people beeing able to comment on best
practices.  I personally would say: Any of the methods you
mentioned above that recreates the same resulting and working
package, runs the testsuite successfully is fine.
 
> Then I was able to run the test, but I ran into another problem:
> although all the selected test passed when run on sid, some of them
> failed when run in the pbuilder. I will investigate this when the
> test suite is fully operational.

There are several options:
 - Tests that might try to connect to online resources will fail.
 - Tests that might need some extra packages you have installed
   on your system by chance might fail - you need to add these
   packages as Build-Depends

> So I decided to remove the incriminated tests...

In the first case this is the correct way to do in the second
I would add the Build-Depends.
 
> We (upstream) will (try to) improve the test suite for the next
> release so that all tests are relevants.

:-)
 
> 2) autopkgtest
> 
> I also explore a little bit the autopkgtest solution, which looks
> great for automatically checking the installed package.
> 
> In this recent post from the autopkgtest maintainers [1], it is
> advised to first read [2], which gives you all the information you
> need.
> Another good, but not complete, documentationto can be found in [3].
> 
> I added the first test in debian/tests. It is the test I usually do
> locally before committing on the debian-med svn (test the
> installation configuration using camitk-config --config as I
> explained in my previous post).
> 
> I was able to check my script in different ways:
> 1. using adt-run to check the current version available (in my case,
> it was on sid) :
> sudo adt-run --no-built-binaries --built-tree=. --- adt-virt-null
> (as explained in [2])
> In this case the available package camitk 3.2.2 package is checked.
> 
> 2. using adt-run to check the build version (in my case the new one)
> 
> These first two ways are probably the laziest way (no virtual environment).
> I also tested adt-run using lxc [4]. Once I had a lxc VM (called
> camitk-sid in my case) up and ready, I was able to do for instance
> using the package produced by pbuilder:
> 
> sudo adt-run --no-built-binaries *.deb *.dsc --gnupg-home=fresh ---
> adt-virt-lxc camitk-sid
> 
> And here is the results:
> ...
> adt-run: & dsc8t-config:  - - - - - - - - - - results - - - - - -
> dsc8t-config         PASS
> adt-run: & dsc8t-config:  - - - - - - - - - - stdout - - - - - -
> Detected CamiTK version is 3.3.0
> Checking configuration...
> CamiTK 3.3.0 configuration: OK
> adt-run: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ tests done.
> 
> 
> This is really great!

+1

> If I understand [1] correctly this means that once uploaded, the
> camitk package will be checked on http://ci.debian.net/

+1

> I will add more test when time allows (checking the compilation of
> camitk extension).
> 
> For both testing (testsuite and autopkgtest) I had to add xvfb and
> xauth to the dependencies, they are needed by our tests (the config
> and test applications both need an X11 context).

I think this is quite normal for X11 applications.

> 3) CamiTK 3.3.0 ready(?)
> 
> Would this be ok now to upload the new version? I checked the last
> committed changes on sid and in a pbuilder environment + the adt-run

Yes, sounds good.  Please give me a final OK if you want me to sponsor
the package.

> Only two lintian warnings are left:
> W: camitk source: unknown-field-in-dsc testsuite

Use `lintian -i` to find out what might be wrong.

> W: camitk source: newer-standards-version 3.9.5 (current is 3.9.4)
> 
> The first one is due to the new field in control for autpkg:
> XS-Testsuite: autopkgtest
> Which I suppose is ok (?) considering I added the X-TestSuite flag
> in debian/control in order to activate autopkgtest

Yes!

> For this second one, I let Andreas decide.

You should simply bump the Standards-Version if lintian tells you to do
so to adapt to the latest Debian Policy.  In the most cases your package
will need no changes - otherwise lintian will tell you so.  I personally
do simply

     cme fix dpkg-control

(as proposed in Debian Med policy).

> Thanks again for helping us improve this package and encouraging me
> to explore autopkgtest.

:-)

Kind regards

        Andreas.

-- 
http://fam-tille.de