[Debian-med-packaging] Function multiversioning for Phylogeny tools [Fwd: Help to build library in generic form, avx and sse3]

Mon Mar 20 10:29:56 UTC 2017

Dear Andreas,

Thanks a lot. The suggestion in the forwarded email really sounds good. 
I've also added Alexey Kozlov to this thread, since he is the main 
developer of the RAxML re-design that we actually released today:

https://github.com/amkozlov/raxml-ng/releases/tag/0.1.0

So I don't think that we will add this to the old RAxML developed by me, 
but we should definitely add this to the new one.

I do believe that it would be best to open issues in the respective 
github repos such that we don't lose track and I will discuss it 
tomorrow morning with some members of my team.

Thanks for all your work on this,

Alexis

On 15.03.2017 13:32, Andreas Tille wrote:
> Hi Alexandros,
>
> recently I asked you about suggestions what else besides RaXML and ExaML
> might be interesting packaging targets for the Debian Med team.  I
> somehow lost the message (I'm archiving now mine here on the public
> Debian Med developers list ;-)) but you suggested
>
>   ExaBayes: https://github.com/aberer/exabayes
>   RogueNaRok: https://github.com/aberer/RogueNaRok
>   mPTP: https://github.com/Pas-Kapli/mptp
>
> by their influence.  I had a look into the first two.  The second was
> easy and thus a roguenarok package is now in the Debian New queue for
> inspection by ftpmaster.  ExaBayes is a bit more complex and I'm in
> discussion with Andre Aberer (in CC) how to proceed[1].
>
> When trying to package libpll (independently whether it will be used to
> link ExaBayes or not in the final package) I stumbled about the problem
> that in contrast to the documentation no generic library (without sse3
> and avx is created) and asked for help on the usual Debian channels.
>
> I'm quoting below the most interesting answer which is probably
> interesting for several tools of your lab since I remember some past
> discussion about RaXML where also different binaries are created which
> might not be only confusing for users but also could lead to situation
> where it is out of control of the user what version is choosen.
>
> I observed it in wrappers like biopython, bioperl and other suites that
> in all cases only the simple version is called and not the hardware
> enhanced versions are choosen.  I tried to work around this situation by
> providing a wrapper in /usr/bin/DefaultNameOfTheTool which checks the
> hardware and redirects the command to the proper executable but finally
> that's a hack I'd like to get rid of rather sooner than later.
>
> So I'd like you to read the mail below about function multi-versioning
> and consider to take this technique over into your source code to
> support all users with sufficient hardware profiting from the maximum
> speed they could gain from their hardware.  If you want me to open
> issues in Github for the single projects I can do so but at first I
> wanted to make you aware about this in general since it is not only a
> single software affected and you might like to coordinate things in your
> lab.
>
> Kind regards
>
>         Andreas.
>
> [1] https://github.com/aberer/exabayes/issues/4
>
> ----- Forwarded message from Christian Seiler <christian at iwakd.de> -----
>
> Date: Tue, 14 Mar 2017 16:03:01 +0100
> From: Christian Seiler <christian at iwakd.de>
> To: Andreas Tille <andreas at an3as.eu>, Debian Mentors List <debian-mentors at lists.debian.org>, Debian Med Packaging Team <debian-med-packaging at lists.alioth.debian.org>
> Subject: Re: Help to build library in generic form, avx and sse3
>
> On 03/14/2017 03:46 PM, Andreas Tille wrote:
>> I've started packaging Phylogenetic Likelihood Library[1].  Since it
>> makes heavy use of amd64 features it comes with specific support of AVX
>> and SSE3.  My plan is to provide binary packages amd64 only named
>> libpll-avx1 and libpll-sse3-1 with the according features plus a generic
>> library libpll-generic1 for all architectures.  Upstream supports the
>> creation of separate avx and sse3 libs out of the box but I failed to
>> create the generic version.  So I have two questions:
>>
>>   1. Could anybody please have a look at the automake stuff to
>>      enable the build of the generic lib in addition to the other
>>      two.  I tried several switches but failed. :-(
>>
>>   2. What do you think about the plan to support specific hardware
>>      features in separate binary packages?
>
> GCC from version 6 (which is in Debian Stretch) supports function
> multi-versioning (and GCC from 4.8 onwards, which is even in Jessie,
> supports a subset of that), which allows you to do the following:
>
>  - have a function with generic C/C++ code be compiled multiple
>    times in different variants, and have the most optimal variant
>    be selected at runtime (requires GCC 6),
>
>    e.g.
>
>    __attribute__((target_clones("avx2","sse3","default")))
>    double foo(double a, double b) {
>      return a + b;
>    }
>
>  - manually write different versions of the function and mark
>    them accordingly (requires GCC 4.8)
>
>    __attribute__((target("default")))
>    double foo(double a, double b) {
>      return a + b;
>    }
>
>    __attribute__((target("sse3")))
>    double foo(double a, double b) {
>      SOME_FANCY_SSE3_CODE;
>    }
>
>    __attribute__((target("avx2")))
>    double foo(double a, double b) {
>      SOME_FANCY_AVX2_CODE;
>    }
>
> So from a purely technical perspective I think the best solution
> would probably be to work with upstream to allow them to support
> FMV properly - and then you only need to compile a single library
> version that will work everywhere, but will select the optimal
> algorithm depending on the machine it's run - win/win.
>
> Further reading:
> https://lwn.net/Articles/691932/
>
> Regards,
> Christian
>
>
> ----- End forwarded message -----
>

-- 
Prof. Dr. Alexandros (Alexis) Stamatakis

Phone:  +49-6221-533-240
Fax:    +49-6221-533-298
E-Mail: Alexandros.Stamatakis at h-its.org
WWW:    www.h-its.org

HITS gGmbH
Schloss-Wolfsbrunnenweg 35
D-69118 Heidelberg
Amtsgericht Mannheim / HRB 337446
Managing Director: Dr. Gesa Schönberger
Scientific Director: Prof. Dr. Michael Strube