[Debian-med-packaging] Bug#775621: python-biopython: FTBFS in jessie: Tests failures

[Alexandros Stamatakis]

Dear All,

I am not exactly sure what you need, do you want the Pthreads-version to 
automatically set the number of threads to 2 if the -T option is not 
specified?

If that's the case I can easily fix this.

Alexis

On 18.01.2015 19:38, Andreas Tille wrote:
> [Alexandros, please read below in how far raxml affects BioPython test
>   suite and a proposal what to do at the very end]
>
> Hi Lucas,
>
> I just finished my tests and noticed the following.  After setting
> up a fresh Jessie chroot I was able to reproduce the problem.  However,
> some debugging showed that I forgot to
>
>      sudo mount proc ${CHROTDIR}/proc -t proc
>
> after this the package was building nicely.  Since I guess you have
> setup your chroot properly I do not think that this would be a problem
> on your side but I'm mentioning it anyway.
>
> See more comments below.
>
> On Sun, Jan 18, 2015 at 03:51:43PM +0100, Lucas Nussbaum wrote:
>> On 18/01/15 at 23:50 +1100, Stuart Prescott wrote:
>>> Control: tag -1 unreproducible
>>>
>>> The actual failure is:
>>>
>>> test_raxml_tool ... FAIL
>>>
>>> […]
>>>
>>> ======================================================================
>>> ERROR: test_raxml (test_raxml_tool.AppTests)
>>> Run RAxML using the wrapper.
>>> ----------------------------------------------------------------------
>>> Traceback (most recent call last):
>>>    File "/«BUILDDIR»/python-
>>> biopython-1.64+dfsg/.pybuild/pythonX.Y_2.7/build/Tests/test_raxml_tool.py",
>>> line 45, in test_raxml
>>>      out, err = cmd()
>>>    File "/«BUILDDIR»/python-
>>> biopython-1.64+dfsg/.pybuild/pythonX.Y_2.7/build/Bio/Application/__init__.py",
>>> line 513, in __call__
>>>      stdout_str, stderr_str)
>>> ApplicationError: Non-zero return code 255 from 'raxmlHPC -m PROTCATWAG -n
>>> test -p 10000 -s Phylip/interlaced2.phy'
>>>
>>> ----------------------------------------------------------------------
>>>
>>> I cannot reproduce this failure in a jessie chroot. I'm unable to cause
>>> raxmlHPC to exit(-1) here when I try (although it certainly has plenty of
>>> places where it can do that in its code).
>>
>> Trying to run raxml manually, I get:
>>
>> (jessie-amd64-sbuild)user at ip-172-31-5-2:/tmp/python-biopython-1.64+dfsg/Tests$ raxmlHPC -m PROTCATWAG -n test -p 10000 -s Phylip/interlaced2.phy
>> Use raxml with SSE3 support (1 cpus)
>>
>> The number of threads is currently set to 1
>> Specify the number of threads to run via -T numberOfThreads
>> NumberOfThreads must be set to an integer value greater than 1
>>
>> (jessie-amd64-sbuild)user at ip-172-31-5-2:/tmp/python-biopython-1.64+dfsg/Tests$ echo $?
>> 255
>>
>>
>> However on my laptop, I get:
>>
>> *** lucas at grr:/tmp/python-biopython-1.64+dfsg/Tests$ raxmlHPC -m PROTCATWAG -n test -p 10000 -s Phylip/interlaced2.phy
>> Use raxml with AVX support (2 cpus)
>>
>> This is the RAxML Master Pthread
>>
>> This is RAxML Worker Pthread Number: 1
>>
>>
>> So it seems that raxml tries to guess the fastest possible implementation based
>> on CPU capabilities. /proc/cpuinfo on EC2 does not include AVX, so it fallbacks
>> to a SSE3 implementation, that requires specifying the number of threads.
>> (it works if I add -T 2)
>
> You might like to have a look into /usr/bin/raxmlHPC[1] which tries to
> detect the hardware in a quite primitive manner.  It was suggested and
> tested in practice by a user.  The problematic thing is that raxmlHPC
> either needs -T option for the "more powerful" architectures or does not
> allow this option, which makes different executables not fully
> compatible.  It might be that the wrapper script is rarely tested and
> does not work on all architectures properly.  The only safe way to deal
> with this would be to deactivate the test in the build process.
>
>> Ideally, there would be a sane default for each RAxML implementation.
>
> That's true and I will talk to raxml upstream about this - however,
> that's to late for Jessie.
>
>> It can easily be argued that this is not RC, given that it should be possible
>> to build the package on a machine where SSE3 is not the default raxml
>> implementation.
>
> Yes, that's what I'd suggest.  Alternatively the fix would be to drop
> the test,
>
>> Another option could be to add '-T 2' to the raxml command-line, but I haven't
>> checked what happens if -T is specified on an implementation that does not
>> support threads.  Or just switch to using raxmlHPC-PTHREADS, but then it
>> defeats the purpose of the test...
>
> As I said. above:  If you force '-T 2' (at least) the last fallback
> option will break.  Raxml upstream should rather make sure it will be
> accepted in all executables and print a warning if it is simply ignored.
>
> Lucas, as the reporter of this bug, would your agree that it is not RC?
>
> Kind regards
>
>         Andreas.
>
> [1] http://anonscm.debian.org/viewvc/debian-med/trunk/packages/raxml/trunk/debian/bin/raxmlHPC?view=markup
>

-- 
Alexandros (Alexis) Stamatakis

Research Group Leader, Heidelberg Institute for Theoretical Studies
Full Professor, Dept. of Informatics, Karlsruhe Institute of Technology
Adjunct Professor, Dept. of Ecology and Evolutionary Biology, University
of Arizona at Tucson

www.exelixis-lab.org