[Debian-med-packaging] Function multiversioning for Phylogeny tools [Fwd: Help to build library in generic form, avx and sse3]

Mon Mar 20 14:37:26 UTC 2017

Hi Alexis,

On Mon, Mar 20, 2017 at 11:29:56AM +0100, Alexandros Stamatakis wrote:
> Dear Andreas,
> 
> Thanks a lot. The suggestion in the forwarded email really sounds good. I've
> also added Alexey Kozlov to this thread, since he is the main developer of
> the RAxML re-design that we actually released today:
> 
> https://github.com/amkozlov/raxml-ng/releases/tag/0.1.0

I've added some preliminary packaging stuff for raxml-ng to the Debian
Med packaging Git.  Its featuring the same habit to include libpll as
Git submodule which is not very nice for packagers.  I think I'll sort
out the libpll build issue regarding function multiversioning first.  To
do so I tried to open an issue for libpll but the only way I found under
support is writing to the pll-users googlegroup.  While I do not think
that this is a convenient way to report issues (some issue tracker on
some Git hosting platform would come to mind as a better alternative) I
created a topic named "Please support function multiversioning" there
and hope it fullfills its purpose.

> So I don't think that we will add this to the old RAxML developed by me, but
> we should definitely add this to the new one.
> 
> I do believe that it would be best to open issues in the respective github
> repos such that we don't lose track and I will discuss it tomorrow morning
> with some members of my team.

I'll do so.

> Thanks for all your work on this,

You are welcome - thanks for your work on writing high quality
scientific free software. 

Kind regards

     Andreas.

> On 15.03.2017 13:32, Andreas Tille wrote:
> >Hi Alexandros,
> >
> >recently I asked you about suggestions what else besides RaXML and ExaML
> >might be interesting packaging targets for the Debian Med team.  I
> >somehow lost the message (I'm archiving now mine here on the public
> >Debian Med developers list ;-)) but you suggested
> >
> >  ExaBayes: https://github.com/aberer/exabayes
> >  RogueNaRok: https://github.com/aberer/RogueNaRok
> >  mPTP: https://github.com/Pas-Kapli/mptp
> >
> >by their influence.  I had a look into the first two.  The second was
> >easy and thus a roguenarok package is now in the Debian New queue for
> >inspection by ftpmaster.  ExaBayes is a bit more complex and I'm in
> >discussion with Andre Aberer (in CC) how to proceed[1].
> >
> >When trying to package libpll (independently whether it will be used to
> >link ExaBayes or not in the final package) I stumbled about the problem
> >that in contrast to the documentation no generic library (without sse3
> >and avx is created) and asked for help on the usual Debian channels.
> >
> >I'm quoting below the most interesting answer which is probably
> >interesting for several tools of your lab since I remember some past
> >discussion about RaXML where also different binaries are created which
> >might not be only confusing for users but also could lead to situation
> >where it is out of control of the user what version is choosen.
> >
> >I observed it in wrappers like biopython, bioperl and other suites that
> >in all cases only the simple version is called and not the hardware
> >enhanced versions are choosen.  I tried to work around this situation by
> >providing a wrapper in /usr/bin/DefaultNameOfTheTool which checks the
> >hardware and redirects the command to the proper executable but finally
> >that's a hack I'd like to get rid of rather sooner than later.
> >
> >So I'd like you to read the mail below about function multi-versioning
> >and consider to take this technique over into your source code to
> >support all users with sufficient hardware profiting from the maximum
> >speed they could gain from their hardware.  If you want me to open
> >issues in Github for the single projects I can do so but at first I
> >wanted to make you aware about this in general since it is not only a
> >single software affected and you might like to coordinate things in your
> >lab.
> >
> >Kind regards
> >
> >        Andreas.
> >
> >[1] https://github.com/aberer/exabayes/issues/4
> >
> >----- Forwarded message from Christian Seiler <christian at iwakd.de> -----
> >
> >Date: Tue, 14 Mar 2017 16:03:01 +0100
> >From: Christian Seiler <christian at iwakd.de>
> >To: Andreas Tille <andreas at an3as.eu>, Debian Mentors List <debian-mentors at lists.debian.org>, Debian Med Packaging Team <debian-med-packaging at lists.alioth.debian.org>
> >Subject: Re: Help to build library in generic form, avx and sse3
> >
> >On 03/14/2017 03:46 PM, Andreas Tille wrote:
> >>I've started packaging Phylogenetic Likelihood Library[1].  Since it
> >>makes heavy use of amd64 features it comes with specific support of AVX
> >>and SSE3.  My plan is to provide binary packages amd64 only named
> >>libpll-avx1 and libpll-sse3-1 with the according features plus a generic
> >>library libpll-generic1 for all architectures.  Upstream supports the
> >>creation of separate avx and sse3 libs out of the box but I failed to
> >>create the generic version.  So I have two questions:
> >>
> >>  1. Could anybody please have a look at the automake stuff to
> >>     enable the build of the generic lib in addition to the other
> >>     two.  I tried several switches but failed. :-(
> >>
> >>  2. What do you think about the plan to support specific hardware
> >>     features in separate binary packages?
> >
> >GCC from version 6 (which is in Debian Stretch) supports function
> >multi-versioning (and GCC from 4.8 onwards, which is even in Jessie,
> >supports a subset of that), which allows you to do the following:
> >
> > - have a function with generic C/C++ code be compiled multiple
> >   times in different variants, and have the most optimal variant
> >   be selected at runtime (requires GCC 6),
> >
> >   e.g.
> >
> >   __attribute__((target_clones("avx2","sse3","default")))
> >   double foo(double a, double b) {
> >     return a + b;
> >   }
> >
> > - manually write different versions of the function and mark
> >   them accordingly (requires GCC 4.8)
> >
> >   __attribute__((target("default")))
> >   double foo(double a, double b) {
> >     return a + b;
> >   }
> >
> >   __attribute__((target("sse3")))
> >   double foo(double a, double b) {
> >     SOME_FANCY_SSE3_CODE;
> >   }
> >
> >   __attribute__((target("avx2")))
> >   double foo(double a, double b) {
> >     SOME_FANCY_AVX2_CODE;
> >   }
> >
> >So from a purely technical perspective I think the best solution
> >would probably be to work with upstream to allow them to support
> >FMV properly - and then you only need to compile a single library
> >version that will work everywhere, but will select the optimal
> >algorithm depending on the machine it's run - win/win.
> >
> >Further reading:
> >https://lwn.net/Articles/691932/
> >
> >Regards,
> >Christian
> >
> >
> >----- End forwarded message -----
> >
> 
> -- 
> Prof. Dr. Alexandros (Alexis) Stamatakis
> 
> Phone:  +49-6221-533-240
> Fax:    +49-6221-533-298
> E-Mail: Alexandros.Stamatakis at h-its.org
> WWW:    www.h-its.org
> 
> HITS gGmbH
> Schloss-Wolfsbrunnenweg 35
> D-69118 Heidelberg
> Amtsgericht Mannheim / HRB 337446
> Managing Director: Dr. Gesa Schönberger
> Scientific Director: Prof. Dr. Michael Strube
> 
> 

-- 
http://fam-tille.de