[Debian-med-packaging] Bug#806214: Status of build time test suites of tree-puzzle

Thu Sep 5 11:21:09 BST 2019

Hello Andreas,

> Would you mind pinging upstream about this?  I have no idea about their
> release schedule.  Version 5.3~rc16 is out for quite some while but may
> be its the right moment to approach them.

I'll notify them. I agree, even this particular release candidate has
been out for a while. Anything useful or any replies I will also CC or
forward to this mailing list.

Kind regards,
Shayan Doust

On 05/09/2019 06:27, Andreas Tille wrote:
> Hi Shayan,
> 
> On Thu, Sep 05, 2019 at 01:45:36AM +0100, Shayan Doust wrote:
>> Some investigative work. I also compiled a clean copy directly from the
>> upstream website to prevent any sort of contamination and to rule out
>> possible upstream fault and then a packaging build attempt (debhelper 11).
> 
> Thanks a lot for diving into this.
>  
>>> I have some gut feeling that the test files for comparison do not
>>> really fit the proper result.
>>
>> This may be the case, but I am having a hard time getting the same
>> output as
>> https://buildd.debian.org/status/fetch.php?pkg=tree-puzzle&arch=all&ver=5.2-11&stamp=1539685923&raw=0
>>
>> I have just ran qp-pure-prot.test as an example:
>>
>> 113,114c86,87
>> < WARNING: Result of chi-square test may not be valid because of
>> < small maximum likelihood frequencies and short sequence length!
>> ---
>>> WARNING: Result of chi-square test may not be valid because of small
>>> maximum likelihood frequencies and short sequence length!
>>
>> * Diffs like this indicate formatting faults which are easy to rectify,
>> or remove as a whole.
>>
>> * The first line of the test output file contains TEST-PUZZLE and the
>> version number. This version number (5.3.rc16) is not visible in the
>> upstream expected output file, so diff will of course display this as a
>> delta.
>>
>> * There is a higher verbosity of paragraphed description / output in the
>> test's newly-generated results file compared to what upstream has. Maybe
>> upstream is running an older / more obselete version of the program?
>>
>> * From what I can see, the numbers seem to match for the most part, but
>> upstream has introduced chi2-value column which is not present in the
>> expected results file.
>>
>> For instance, running qp-pure-bin.test. Here is a small slice of my output:
>>
>> 194c169
>> < (bipartition with sequences in input order : number of times seen (and
>> ratio))
>> ---
>>> (bipartition with sequences in input order : number of times seen)
>> 196,197c171,172
>> <  *..** :  1000 (1.000)
>> <  *...* :  1000 (1.000)
>> ---
>>>  *..**  :  1000
>>>  *...*  :  1000
>> 201c176
>> < (bipartition with sequences in input order : number of times seen (and
>> ratio))
>> ---
>>> (bipartition with sequences in input order : number of times seen)
>>
>> Ratios seem to have been added to the test. Scanning through all the
>> other test binaries and their outputs, my values seem to match up to
>> what is expected regardless of the issues above.
>>
>> So next I decided to see when the tests directory was added. This
>> happened to be version 5.2. Quickly testing a couple of test binaries,
>> they run successfully. It seems like to me that the expected test
>> results file introduced in 5.2 were just never updated to 5.3~rc16.
> 
> This matches perfectly my suspicion after a way less deep inspection.
> Thanks for confirming.
> 
>> I am
>> not sure why your build log is different in terms of value. Maybe
>> upstream modified the current release candidate and version (some do
>> this). I will keep this updated with anything else that I spot or comes
>> to mind.
> 
> Would you mind pinging upstream about this?  I have no idea about their
> release schedule.  Version 5.3~rc16 is out for quite some while but may
> be its the right moment to approach them.
>  
>> Kind regards,
> 
> Thanks again for your very valuable contribution
> 
>      Andreas.
>  
>> On Mon, 8 Jul 2019 21:26:02 +0200 Andreas Tille <andreas at an3as.eu> wrote:
>>> Hi,
>>>
>>> after switching tree-puzzle debhelper level to 9  I was cheating around
>>> the build time test suite via
>>>
>>> override_dh_auto_test:
>>> 	# unfortunately most tests are failing for the moment
>>> 	# the issue is documented in
>>> 	#   debian/patches/patch_test_results.patch
>>> 	# and needs to be discussed with upstream
>>> 	dh_auto_test || true
>>>
>>> The rationale was that just by switching the debhelper level the build
>>> time test suite was run at all.  Most probably it was failing all the
>>> time before and simply nobody realised this.  To sort this out we need
>>> to talk to upstream.  The issue is documented in bug #806214 (bug in
>>> CC).
>>>
>>> I now bumped the upstream source in Git to the latest upstream release
>>> candidate.  Since this had not changed quite some time I assume upstream
>>> is not very rapidly pushing a final release.  However, this might be the
>>> right point in time to sort things out.
>>>
>>> If you check the build log of 5.2-11 at
>>>
>>>    https://buildd.debian.org/status/fetch.php?pkg=tree-puzzle&arch=all&ver=5.2-11&stamp=1539685923&raw=0
>>>
>>> you can find
>>>
>>> ...
>>> dh_auto_test || true
>>>  make -j1 check VERBOSE=1
>>> make[2]: Entering directory '/<<PKGBUILDDIR>>'
>>> Making check in src
>>> make[3]: Entering directory '/<<PKGBUILDDIR>>/src'
>>> make[3]: Leaving directory '/<<PKGBUILDDIR>>/src'
>>> Making check in doc
>>> make[3]: Entering directory '/<<PKGBUILDDIR>>/doc'
>>> make[4]: Entering directory '/<<PKGBUILDDIR>>/doc'
>>> make[4]: Nothing to be done for 'check-am'.
>>> make[4]: Leaving directory '/<<PKGBUILDDIR>>/doc'
>>> make[3]: Leaving directory '/<<PKGBUILDDIR>>/doc'
>>> Making check in data
>>> make[3]: Entering directory '/<<PKGBUILDDIR>>/data'
>>> make[3]: Nothing to be done for 'check'.
>>> make[3]: Leaving directory '/<<PKGBUILDDIR>>/data'
>>> Making check in tests
>>> make[3]: Entering directory '/<<PKGBUILDDIR>>/tests'
>>> make  check-TESTS
>>> make[4]: Entering directory '/<<PKGBUILDDIR>>/tests'
>>> make[5]: Entering directory '/<<PKGBUILDDIR>>/tests'
>>> SKIP: build-puzzle
>>> FAIL: qp-pure-bin.test
>>> FAIL: qp-pure-nucl.test
>>> FAIL: qp-tn-nucl.test
>>> FAIL: qp-hky-clock-nucl.test
>>> FAIL: qp-hky-rhet-nucl.test
>>> FAIL: qp-hky-rhet-clock-nucl.test
>>> FAIL: qp-pure-prot.test
>>> FAIL: qp-mtrev-prot.test
>>
> 
> 
> 
> 
>> _______________________________________________
>> Debian-med-packaging mailing list
>> Debian-med-packaging at alioth-lists.debian.net
>> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-med-packaging
> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/debian-med-packaging/attachments/20190905/4657dd42/attachment-0001.sig>