[Debian-med-packaging] Any chance to only partly disable tests for camitk (Re: r16503 - in trunk/packages/camitk/trunk/debian: . patches)

[Andreas Tille]

Hi Emmanuel,

On Mon, Mar 24, 2014 at 09:52:26AM +0100, Emmanuel Promayon wrote:
> I was about to send an email to say that the package camitk 3.3.0-1
> was ready to upload, but your vigilance is ahead of me!
> I checked the build in both sid and pbuilder environment, and it seems ok.
> I added a lintian override because one of the example/test data
> comes with a CC-BY-SA license, and I felt the LICENSE file should be
> left in the directory with the file (see also [1], with seems
> reasonable to me).
> Q: Does this seems ok to you? Any advice about this is welcome.

I think an override is OK here.
 
> Another question I have is about old patches.
> Q: can I remove old .diff that are not needed anymore on the svn?

Yes.  These are kept in old tags and even if I tend to remove *very* old
tags because there complaints about the continous growth of the SVN
without any practical use since all uploads are kept on
snapshot.debian.org you can always fetch the patches from there if needed.
So removing unneeded code is always fine.

> Concerning the test:
> >since we are trying to get tests working as good as possible this step
> >seems to be a bit backwards.  Could you please give reasons why you are
> >disabling the tests or is there any chance to run tests only partially?
> >In any case you should document the problem in README.source and perhaps
> >also explain in README.test how the package could be testet.  Moreover
> >providing an autopkgtest control file would be really great.
> 
> First thanks for your remarks and questions, as usual it makes me go
> further into what we have/need.
> We are willing to include the maximum of automatic tests in the
> framework as well as in the packaging.
> 
> Here is a tentative explanation about why I decided to disable the
> testing phase. Starting with release 3.3.0, we (upstream) added some
> automatic testing for each software plugin (we call them
> "extensions"). These tests are using two new applications
> (camitk-testcomponents and camitk-testactions) and are run by ctest
> (the cmake test environment).
> 
> As always, we started with a basic idea, implemented a first draft
> and checked this first step. That is were we are now.
> At the moment we do not think that the test are (all) relevant. This
> is why I decided to avoid the test.
> 
> As I said, I (with both my upstream and packager hats) am willing to
> go a step further.

OK.  Could you please add this information to README.source.  It is just
to keep your fellow Debian maintainers informed in case anyone might
stumble upon this and should not be forced to seek the web for this
piece of information.

> Concerning testing the framework:
> 1. I usually check the package by installing it with dpkg and then
> run the diagnosis tool "camitk-config" (and... there was a bug in
> the application that is fixed by patch
> config-console-redirection.diff)
> Running "camitk-config --config". I compare the output string with
> the expected one (camitk-config prints out version number,
> installation path, number of plugins, and details about the plugin
> names and descriptions). If camitk-config finds the plugins, it
> means they will be available in the GUI.
> Q: is it possible to add this test to automatically run it in the
> packaging process? That would be great!

Sure.  You can bundle this into a script and do

override_dh_auto_test:
	<run_your_script>

Moreover you can add an autopkgtest (see for example the package
python-pysam or the specification[1]).  Charles just mentioned on the
mailing list sadt(1) which might be helpful as well (even if I did not
yet dived into this).
 
> 2. I also check the developer package. We have an application that
> generates skeleton CamiTK extension code from an XML file. There is
> a GUI application that helps the user creates the XML file and a
> command line only application (camitk-cepgenerator) that generates
> the C++ code from the XML file. I test camitk-cepgenerator and then
> check if the generated code builds and is recognize by camitk-config
> as a new extension.
> This is done by (roughly):
> $ cd /tmp
> $ rm -rf /tmp/testcepgenerator && mkdir /tmp/testcepgenerator
> $ camitk-cepgenerator -f input.xml -d /tmp/testcepgenerator
> $ cd /tmp/testcepgenerator
> $ mkdir build
> $ cd build
> $ cmake ../generatedsourcecodedir/
> $ make -j2
> $ camitk-config -c | grep "\[W\]"
> This should answer with a text that shows that the extension is
> compiled and recognized by the framework.
> At the moment there are four types of XML test files.
> Q: same as above: is it possible to add this test and automatically run it?

Everything that is scriptable can be used as test.  You can do as
I wrote above and add another script doing the later.
 
> Q: What do you think I should put in README.source and in
> README.test? Do you think the "too new to make sense" explanation is
> acceptable?

Well, you are upstream and you will now best how to acceptably test your
code.  So if your answer is "yes it is" my answer will not be different.

> In the meanwhile, I am going to look at autopkgtest (which seems
> promising, and seems to answer some of my questions!). Do you have
> any idea where I can find an example (especially if this can be a
> mix between disabling automatic ctest and enabling global package
> testing)?

See above for an example which I personally find more speaking and
the link to the spec[1].

Hope this helps 

    Andreas.

[1] http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=blob_plain;f=doc/README.package-tests;hb=HEAD

-- 
http://fam-tille.de