Bug#842513: vlc: immediate crash on launch on powerpc

[Lennart Sorensen]

On Fri, Nov 04, 2016 at 10:52:05AM +0100, Sebastian Ramacher wrote:
> On 2016-11-03 13:58:58, Lennart Sorensen wrote:
> > On Tue, Nov 01, 2016 at 01:42:17PM -0400, Lennart Sorensen wrote:
> > > Well that looks like it only adds it for libvlccore.
> > 
> > So at least something like this is needed, but I think the VLCCORE is
> > wrong too, and maybe the deinterlace has to be moved to only merge.c
> > rather than all of deinterlace.
> > 
> > --- configure.ac.orig   2016-11-03 13:56:25.826878272 -0400
> > +++ configure.ac        2016-11-03 13:56:55.558878272 -0400
> > @@ -1431,7 +1431,6 @@
> >        ac_cv_c_altivec="no"
> >      ])
> >    ])
> > -  VLC_RESTORE_FLAGS
> >    AS_IF([test "${ac_cv_c_altivec}" != "no"], [
> >      CPPFLAGS="${CPPFLAGS} ${ac_cv_c_altivec}"
> >      AC_DEFINE(CAN_COMPILE_C_ALTIVEC, 1, [Define to 1 if C AltiVec extensions are available.])
> > @@ -1441,6 +1440,7 @@
> >      have_altivec="yes"
> >    ])
> >    AC_CHECK_HEADERS(altivec.h)
> > +  VLC_RESTORE_FLAGS
> >  
> >    VLC_SAVE_FLAGS
> >    LDFLAGS="${LDFLAGS} -Wl,-framework,vecLib"
> > 
> > The CPPFLAGS are being clobbered with the -maltivec, which is only needed
> > when testing altivec.h, and should be removed again after that.
> > 
> > That removes the -maltivec that was showing up everywhere at least, and
> > now it is only in a few places, although perhaps still a few too many.
> 
> Please get this (and any follow-up patches) integrated upstream. See
> https://wiki.videolan.org/Sending_Patches/ for details.

Well it's not the complete solution yet.

I have it down to only deinterlace being miscompiled (it auto vectorized
Generic* functions with altivec instructions, which is not good).

At the moment it looks like either the altivec stuff has to be #ifdef
out and compiled in a separeta pass like the yuv _altivec_ plugin, or
gcc #pragma has to be used to only enable altivec in the one function
(which I am trying now).  #pragma seems a tad ugly though.

I suspect gcc didn't always generate vector code automatically when
-maltivec was passed, but these days it does, even at -O2 it seems,
even though all the places I can find in the documentation says only -O3
enables vectorization.

-- 
Len Sorensen