[Debian-med-packaging] Function multiversioning for Phylogeny tools [Fwd: Help to build library in generic form, avx and sse3]

Mon Mar 20 10:31:13 UTC 2017

and by the way I've added Andre's current email,

alexis

On 20.03.2017 11:29, Alexandros Stamatakis wrote:
> Dear Andreas,
>
> Thanks a lot. The suggestion in the forwarded email really sounds good.
> I've also added Alexey Kozlov to this thread, since he is the main
> developer of the RAxML re-design that we actually released today:
>
> https://github.com/amkozlov/raxml-ng/releases/tag/0.1.0
>
> So I don't think that we will add this to the old RAxML developed by me,
> but we should definitely add this to the new one.
>
> I do believe that it would be best to open issues in the respective
> github repos such that we don't lose track and I will discuss it
> tomorrow morning with some members of my team.
>
> Thanks for all your work on this,
>
> Alexis
>
>
> On 15.03.2017 13:32, Andreas Tille wrote:
>> Hi Alexandros,
>>
>> recently I asked you about suggestions what else besides RaXML and ExaML
>> might be interesting packaging targets for the Debian Med team.  I
>> somehow lost the message (I'm archiving now mine here on the public
>> Debian Med developers list ;-)) but you suggested
>>
>>   ExaBayes: https://github.com/aberer/exabayes
>>   RogueNaRok: https://github.com/aberer/RogueNaRok
>>   mPTP: https://github.com/Pas-Kapli/mptp
>>
>> by their influence.  I had a look into the first two.  The second was
>> easy and thus a roguenarok package is now in the Debian New queue for
>> inspection by ftpmaster.  ExaBayes is a bit more complex and I'm in
>> discussion with Andre Aberer (in CC) how to proceed[1].
>>
>> When trying to package libpll (independently whether it will be used to
>> link ExaBayes or not in the final package) I stumbled about the problem
>> that in contrast to the documentation no generic library (without sse3
>> and avx is created) and asked for help on the usual Debian channels.
>>
>> I'm quoting below the most interesting answer which is probably
>> interesting for several tools of your lab since I remember some past
>> discussion about RaXML where also different binaries are created which
>> might not be only confusing for users but also could lead to situation
>> where it is out of control of the user what version is choosen.
>>
>> I observed it in wrappers like biopython, bioperl and other suites that
>> in all cases only the simple version is called and not the hardware
>> enhanced versions are choosen.  I tried to work around this situation by
>> providing a wrapper in /usr/bin/DefaultNameOfTheTool which checks the
>> hardware and redirects the command to the proper executable but finally
>> that's a hack I'd like to get rid of rather sooner than later.
>>
>> So I'd like you to read the mail below about function multi-versioning
>> and consider to take this technique over into your source code to
>> support all users with sufficient hardware profiting from the maximum
>> speed they could gain from their hardware.  If you want me to open
>> issues in Github for the single projects I can do so but at first I
>> wanted to make you aware about this in general since it is not only a
>> single software affected and you might like to coordinate things in your
>> lab.
>>
>> Kind regards
>>
>>         Andreas.
>>
>> [1] https://github.com/aberer/exabayes/issues/4
>>
>> ----- Forwarded message from Christian Seiler <christian at iwakd.de> -----
>>
>> Date: Tue, 14 Mar 2017 16:03:01 +0100
>> From: Christian Seiler <christian at iwakd.de>
>> To: Andreas Tille <andreas at an3as.eu>, Debian Mentors List
>> <debian-mentors at lists.debian.org>, Debian Med Packaging Team
>> <debian-med-packaging at lists.alioth.debian.org>
>> Subject: Re: Help to build library in generic form, avx and sse3
>>
>> On 03/14/2017 03:46 PM, Andreas Tille wrote:
>>> I've started packaging Phylogenetic Likelihood Library[1].  Since it
>>> makes heavy use of amd64 features it comes with specific support of AVX
>>> and SSE3.  My plan is to provide binary packages amd64 only named
>>> libpll-avx1 and libpll-sse3-1 with the according features plus a generic
>>> library libpll-generic1 for all architectures.  Upstream supports the
>>> creation of separate avx and sse3 libs out of the box but I failed to
>>> create the generic version.  So I have two questions:
>>>
>>>   1. Could anybody please have a look at the automake stuff to
>>>      enable the build of the generic lib in addition to the other
>>>      two.  I tried several switches but failed. :-(
>>>
>>>   2. What do you think about the plan to support specific hardware
>>>      features in separate binary packages?
>>
>> GCC from version 6 (which is in Debian Stretch) supports function
>> multi-versioning (and GCC from 4.8 onwards, which is even in Jessie,
>> supports a subset of that), which allows you to do the following:
>>
>>  - have a function with generic C/C++ code be compiled multiple
>>    times in different variants, and have the most optimal variant
>>    be selected at runtime (requires GCC 6),
>>
>>    e.g.
>>
>>    __attribute__((target_clones("avx2","sse3","default")))
>>    double foo(double a, double b) {
>>      return a + b;
>>    }
>>
>>  - manually write different versions of the function and mark
>>    them accordingly (requires GCC 4.8)
>>
>>    __attribute__((target("default")))
>>    double foo(double a, double b) {
>>      return a + b;
>>    }
>>
>>    __attribute__((target("sse3")))
>>    double foo(double a, double b) {
>>      SOME_FANCY_SSE3_CODE;
>>    }
>>
>>    __attribute__((target("avx2")))
>>    double foo(double a, double b) {
>>      SOME_FANCY_AVX2_CODE;
>>    }
>>
>> So from a purely technical perspective I think the best solution
>> would probably be to work with upstream to allow them to support
>> FMV properly - and then you only need to compile a single library
>> version that will work everywhere, but will select the optimal
>> algorithm depending on the machine it's run - win/win.
>>
>> Further reading:
>> https://lwn.net/Articles/691932/
>>
>> Regards,
>> Christian
>>
>>
>> ----- End forwarded message -----
>>
>

-- 
Prof. Dr. Alexandros (Alexis) Stamatakis

Phone:  +49-6221-533-240
Fax:    +49-6221-533-298
E-Mail: Alexandros.Stamatakis at h-its.org
WWW:    www.h-its.org

HITS gGmbH
Schloss-Wolfsbrunnenweg 35
D-69118 Heidelberg
Amtsgericht Mannheim / HRB 337446
Managing Director: Dr. Gesa Schönberger
Scientific Director: Prof. Dr. Michael Strube