Bug#699722: src:libav: x264 decoding crashes

[Reinhard Tartler]

On Tue, Feb 5, 2013 at 2:10 AM, Dmitry Smirnov <onlyjob at member.fsf.org> wrote:
> This is happening only when I build xpra with the following packages from experimental:
>
>         libavcodec-dev libavutil-dev libswscale-dev libswscale2
>
> Merely upgrading libswscale2 doesn't break x264 encoding.

OK, I've checked with libav upstream, but I'm a bit in hurry right
now. Please have a look at the irc chatlog. In short, xpra is buggy,
and Anton (email in CC) is even proposing a patch to fix xpra:


07:20 <siretart> elenril: can you make any sense out of
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=699722#17 ? -
                 Thread 2, frame #6 indicates that
avcodec_get_frame_defaults() frees (via av_freep) a pointer that it
                 shouldn't free.
07:22 <siretart> elenril: submitter claims that downgrading to 0.8.5
avoids the crash. now I wonder if the fault is on
                 our, or on the xpra side
07:26 <elenril> it's probably old x264 version with a new libav version
07:26 <elenril> x264 used to abuse our api, that particular way of
abuse got broken
07:30 <siretart> elenril: on that case, we probably should require a
new-enough x264 version in configure. or at least warn about it. what
do you think?
07:31 <siretart> the package in question is running libx264 0.129.2238
07:31 <siretart> is this too old?
07:31 <siretart> elenril: ^^
07:33 <siretart> build 129 is the newest version available, so I'm not
entirely convinced by your theory
07:34 <elenril> see 6e68ab73908f339cdd91c40943fef46fd1f832fa in x264
07:35 <elenril> and i don't see how _us_ requiring a specific libx264
solves anything
07:35 <elenril> the problem is x264 calling libav
07:35 <elenril> not libav calling x264
07:35 <elenril> and that package appears to contain embedded x264
07:35 <elenril> isn't that evil?
07:39 <elenril> hmm...no
07:39 <elenril> but it abuses our api in exactly the same way
07:40 <elenril> it has an uninitialized AVFrame on stack
07:40 <siretart> looking
07:42 <siretart> elenril: hm, the submitter does use a libx264 that
has this patch included
07:43 <siretart> elenril: or do you imply that even with
6e68ab73908f339cdd91c40943fef46fd1f832fa, libx264 is still bugged?
07:43 <elenril> see the last three lines i said
07:44 <siretart> ok, what does "it has an uninitialized AVFrame on
stack" now mean? is the application responsible for having it properly
allocated? don't we have
                 an allocation API that DTRT?
07:45 <elenril> http://paste.debian.net/231849/
07:45 <elenril> here's a fixed version
07:46 <siretart> fixed version of what?
07:46 <elenril> of the broken file in xpra
07:46 <elenril> let me make a patch
07:48 <elenril> http://paste.debian.net/231850/

Cheers,

-- 
regards,
    Reinhard