[Debian-med-packaging] Bug#806214: Status of build time test suites of tree-puzzle

[Andreas Tille]

Hi Shayan,

On Thu, Sep 05, 2019 at 01:45:36AM +0100, Shayan Doust wrote:
> Some investigative work. I also compiled a clean copy directly from the
> upstream website to prevent any sort of contamination and to rule out
> possible upstream fault and then a packaging build attempt (debhelper 11).

Thanks a lot for diving into this.
 
> > I have some gut feeling that the test files for comparison do not
> > really fit the proper result.
> 
> This may be the case, but I am having a hard time getting the same
> output as
> https://buildd.debian.org/status/fetch.php?pkg=tree-puzzle&arch=all&ver=5.2-11&stamp=1539685923&raw=0
> 
> I have just ran qp-pure-prot.test as an example:
> 
> 113,114c86,87
> < WARNING: Result of chi-square test may not be valid because of
> < small maximum likelihood frequencies and short sequence length!
> ---
> > WARNING: Result of chi-square test may not be valid because of small
> > maximum likelihood frequencies and short sequence length!
> 
> * Diffs like this indicate formatting faults which are easy to rectify,
> or remove as a whole.
> 
> * The first line of the test output file contains TEST-PUZZLE and the
> version number. This version number (5.3.rc16) is not visible in the
> upstream expected output file, so diff will of course display this as a
> delta.
> 
> * There is a higher verbosity of paragraphed description / output in the
> test's newly-generated results file compared to what upstream has. Maybe
> upstream is running an older / more obselete version of the program?
> 
> * From what I can see, the numbers seem to match for the most part, but
> upstream has introduced chi2-value column which is not present in the
> expected results file.
> 
> For instance, running qp-pure-bin.test. Here is a small slice of my output:
> 
> 194c169
> < (bipartition with sequences in input order : number of times seen (and
> ratio))
> ---
> > (bipartition with sequences in input order : number of times seen)
> 196,197c171,172
> <  *..** :  1000 (1.000)
> <  *...* :  1000 (1.000)
> ---
> >  *..**  :  1000
> >  *...*  :  1000
> 201c176
> < (bipartition with sequences in input order : number of times seen (and
> ratio))
> ---
> > (bipartition with sequences in input order : number of times seen)
> 
> Ratios seem to have been added to the test. Scanning through all the
> other test binaries and their outputs, my values seem to match up to
> what is expected regardless of the issues above.
> 
> So next I decided to see when the tests directory was added. This
> happened to be version 5.2. Quickly testing a couple of test binaries,
> they run successfully. It seems like to me that the expected test
> results file introduced in 5.2 were just never updated to 5.3~rc16.

This matches perfectly my suspicion after a way less deep inspection.
Thanks for confirming.

> I am
> not sure why your build log is different in terms of value. Maybe
> upstream modified the current release candidate and version (some do
> this). I will keep this updated with anything else that I spot or comes
> to mind.

Would you mind pinging upstream about this?  I have no idea about their
release schedule.  Version 5.3~rc16 is out for quite some while but may
be its the right moment to approach them.
 
> Kind regards,

Thanks again for your very valuable contribution

     Andreas.
 
> On Mon, 8 Jul 2019 21:26:02 +0200 Andreas Tille <andreas at an3as.eu> wrote:
> > Hi,
> > 
> > after switching tree-puzzle debhelper level to 9  I was cheating around
> > the build time test suite via
> > 
> > override_dh_auto_test:
> > 	# unfortunately most tests are failing for the moment
> > 	# the issue is documented in
> > 	#   debian/patches/patch_test_results.patch
> > 	# and needs to be discussed with upstream
> > 	dh_auto_test || true
> > 
> > The rationale was that just by switching the debhelper level the build
> > time test suite was run at all.  Most probably it was failing all the
> > time before and simply nobody realised this.  To sort this out we need
> > to talk to upstream.  The issue is documented in bug #806214 (bug in
> > CC).
> > 
> > I now bumped the upstream source in Git to the latest upstream release
> > candidate.  Since this had not changed quite some time I assume upstream
> > is not very rapidly pushing a final release.  However, this might be the
> > right point in time to sort things out.
> > 
> > If you check the build log of 5.2-11 at
> > 
> >    https://buildd.debian.org/status/fetch.php?pkg=tree-puzzle&arch=all&ver=5.2-11&stamp=1539685923&raw=0
> > 
> > you can find
> > 
> > ...
> > dh_auto_test || true
> >  make -j1 check VERBOSE=1
> > make[2]: Entering directory '/<<PKGBUILDDIR>>'
> > Making check in src
> > make[3]: Entering directory '/<<PKGBUILDDIR>>/src'
> > make[3]: Leaving directory '/<<PKGBUILDDIR>>/src'
> > Making check in doc
> > make[3]: Entering directory '/<<PKGBUILDDIR>>/doc'
> > make[4]: Entering directory '/<<PKGBUILDDIR>>/doc'
> > make[4]: Nothing to be done for 'check-am'.
> > make[4]: Leaving directory '/<<PKGBUILDDIR>>/doc'
> > make[3]: Leaving directory '/<<PKGBUILDDIR>>/doc'
> > Making check in data
> > make[3]: Entering directory '/<<PKGBUILDDIR>>/data'
> > make[3]: Nothing to be done for 'check'.
> > make[3]: Leaving directory '/<<PKGBUILDDIR>>/data'
> > Making check in tests
> > make[3]: Entering directory '/<<PKGBUILDDIR>>/tests'
> > make  check-TESTS
> > make[4]: Entering directory '/<<PKGBUILDDIR>>/tests'
> > make[5]: Entering directory '/<<PKGBUILDDIR>>/tests'
> > SKIP: build-puzzle
> > FAIL: qp-pure-bin.test
> > FAIL: qp-pure-nucl.test
> > FAIL: qp-tn-nucl.test
> > FAIL: qp-hky-clock-nucl.test
> > FAIL: qp-hky-rhet-nucl.test
> > FAIL: qp-hky-rhet-clock-nucl.test
> > FAIL: qp-pure-prot.test
> > FAIL: qp-mtrev-prot.test
> 




> _______________________________________________
> Debian-med-packaging mailing list
> Debian-med-packaging at alioth-lists.debian.net
> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-med-packaging


-- 
http://fam-tille.de