[Debian-med-packaging] Function multiversioning for Phylogeny tools [Fwd: Help to build library in generic form, avx and sse3]

Tue Mar 21 08:11:30 UTC 2017

Hi Andreas,

> I've added some preliminary packaging stuff for raxml-ng to the Debian
> Med packaging Git.  Its featuring the same habit to include libpll as
> Git submodule which is not very nice for packagers.

Not only for packagers, but I don't see a better way of doing it.

> I think I'll sort
> out the libpll build issue regarding function multiversioning first.  To
> do so I tried to open an issue for libpll but the only way I found under
> support is writing to the pll-users googlegroup.  While I do not think
> that this is a convenient way to report issues (some issue tracker on
> some Git hosting platform would come to mind as a better alternative) I
> created a topic named "Please support function multiversioning" there
> and hope it fullfills its purpose.

Yes I saw that, but why can't you open a github issue for LLPLL, Tomas 
did you disable them?

>> So I don't think that we will add this to the old RAxML developed by me, but
>> we should definitely add this to the new one.
>>
>> I do believe that it would be best to open issues in the respective github
>> repos such that we don't lose track and I will discuss it tomorrow morning
>> with some members of my team.
>
> I'll do so.
>
>> Thanks for all your work on this,
>
> You are welcome - thanks for your work on writing high quality
> scientific free software.

:-)

Alexis

>
> Kind regards
>
>      Andreas.
>
>> On 15.03.2017 13:32, Andreas Tille wrote:
>>> Hi Alexandros,
>>>
>>> recently I asked you about suggestions what else besides RaXML and ExaML
>>> might be interesting packaging targets for the Debian Med team.  I
>>> somehow lost the message (I'm archiving now mine here on the public
>>> Debian Med developers list ;-)) but you suggested
>>>
>>>  ExaBayes: https://github.com/aberer/exabayes
>>>  RogueNaRok: https://github.com/aberer/RogueNaRok
>>>  mPTP: https://github.com/Pas-Kapli/mptp
>>>
>>> by their influence.  I had a look into the first two.  The second was
>>> easy and thus a roguenarok package is now in the Debian New queue for
>>> inspection by ftpmaster.  ExaBayes is a bit more complex and I'm in
>>> discussion with Andre Aberer (in CC) how to proceed[1].
>>>
>>> When trying to package libpll (independently whether it will be used to
>>> link ExaBayes or not in the final package) I stumbled about the problem
>>> that in contrast to the documentation no generic library (without sse3
>>> and avx is created) and asked for help on the usual Debian channels.
>>>
>>> I'm quoting below the most interesting answer which is probably
>>> interesting for several tools of your lab since I remember some past
>>> discussion about RaXML where also different binaries are created which
>>> might not be only confusing for users but also could lead to situation
>>> where it is out of control of the user what version is choosen.
>>>
>>> I observed it in wrappers like biopython, bioperl and other suites that
>>> in all cases only the simple version is called and not the hardware
>>> enhanced versions are choosen.  I tried to work around this situation by
>>> providing a wrapper in /usr/bin/DefaultNameOfTheTool which checks the
>>> hardware and redirects the command to the proper executable but finally
>>> that's a hack I'd like to get rid of rather sooner than later.
>>>
>>> So I'd like you to read the mail below about function multi-versioning
>>> and consider to take this technique over into your source code to
>>> support all users with sufficient hardware profiting from the maximum
>>> speed they could gain from their hardware.  If you want me to open
>>> issues in Github for the single projects I can do so but at first I
>>> wanted to make you aware about this in general since it is not only a
>>> single software affected and you might like to coordinate things in your
>>> lab.
>>>
>>> Kind regards
>>>
>>>        Andreas.
>>>
>>> [1] https://github.com/aberer/exabayes/issues/4
>>>
>>> ----- Forwarded message from Christian Seiler <christian at iwakd.de> -----
>>>
>>> Date: Tue, 14 Mar 2017 16:03:01 +0100
>>> From: Christian Seiler <christian at iwakd.de>
>>> To: Andreas Tille <andreas at an3as.eu>, Debian Mentors List <debian-mentors at lists.debian.org>, Debian Med Packaging Team <debian-med-packaging at lists.alioth.debian.org>
>>> Subject: Re: Help to build library in generic form, avx and sse3
>>>
>>> On 03/14/2017 03:46 PM, Andreas Tille wrote:
>>>> I've started packaging Phylogenetic Likelihood Library[1].  Since it
>>>> makes heavy use of amd64 features it comes with specific support of AVX
>>>> and SSE3.  My plan is to provide binary packages amd64 only named
>>>> libpll-avx1 and libpll-sse3-1 with the according features plus a generic
>>>> library libpll-generic1 for all architectures.  Upstream supports the
>>>> creation of separate avx and sse3 libs out of the box but I failed to
>>>> create the generic version.  So I have two questions:
>>>>
>>>>  1. Could anybody please have a look at the automake stuff to
>>>>     enable the build of the generic lib in addition to the other
>>>>     two.  I tried several switches but failed. :-(
>>>>
>>>>  2. What do you think about the plan to support specific hardware
>>>>     features in separate binary packages?
>>>
>>> GCC from version 6 (which is in Debian Stretch) supports function
>>> multi-versioning (and GCC from 4.8 onwards, which is even in Jessie,
>>> supports a subset of that), which allows you to do the following:
>>>
>>> - have a function with generic C/C++ code be compiled multiple
>>>   times in different variants, and have the most optimal variant
>>>   be selected at runtime (requires GCC 6),
>>>
>>>   e.g.
>>>
>>>   __attribute__((target_clones("avx2","sse3","default")))
>>>   double foo(double a, double b) {
>>>     return a + b;
>>>   }
>>>
>>> - manually write different versions of the function and mark
>>>   them accordingly (requires GCC 4.8)
>>>
>>>   __attribute__((target("default")))
>>>   double foo(double a, double b) {
>>>     return a + b;
>>>   }
>>>
>>>   __attribute__((target("sse3")))
>>>   double foo(double a, double b) {
>>>     SOME_FANCY_SSE3_CODE;
>>>   }
>>>
>>>   __attribute__((target("avx2")))
>>>   double foo(double a, double b) {
>>>     SOME_FANCY_AVX2_CODE;
>>>   }
>>>
>>> So from a purely technical perspective I think the best solution
>>> would probably be to work with upstream to allow them to support
>>> FMV properly - and then you only need to compile a single library
>>> version that will work everywhere, but will select the optimal
>>> algorithm depending on the machine it's run - win/win.
>>>
>>> Further reading:
>>> https://lwn.net/Articles/691932/
>>>
>>> Regards,
>>> Christian
>>>
>>>
>>> ----- End forwarded message -----
>>>
>>
>> --
>> Prof. Dr. Alexandros (Alexis) Stamatakis
>>
>> Phone:  +49-6221-533-240
>> Fax:    +49-6221-533-298
>> E-Mail: Alexandros.Stamatakis at h-its.org
>> WWW:    www.h-its.org
>>
>> HITS gGmbH
>> Schloss-Wolfsbrunnenweg 35
>> D-69118 Heidelberg
>> Amtsgericht Mannheim / HRB 337446
>> Managing Director: Dr. Gesa Schönberger
>> Scientific Director: Prof. Dr. Michael Strube
>>
>>
>

-- 
Prof. Dr. Alexandros (Alexis) Stamatakis

Phone:  +49-6221-533-240
Fax:    +49-6221-533-298
E-Mail: Alexandros.Stamatakis at h-its.org
WWW:    www.h-its.org

HITS gGmbH
Schloss-Wolfsbrunnenweg 35
D-69118 Heidelberg
Amtsgericht Mannheim / HRB 337446
Managing Director: Dr. Gesa Schönberger
Scientific Director: Prof. Dr. Michael Strube