Select provider of libav* libraries

[Andreas Cadhalpun]

Hi Reinhard,

On 18.05.2015 12:16, Reinhard Tartler wrote:
> Please excuse my previous unfinished reply, it was sent in accident.

No problem, such things can happen.

> I'm not sure if this post really adds to this discussion, please
> consider it as clarifications to my previous post.

I find it much better than getting no reply, so thanks for following up.

> On May 15, 2015 4:56 PM, "Andreas Cadhalpun"
>> > I think security is not a decisive topic where either project cannot
>> > clearly claim of having a clearly better handle.
>>
>> I disagree: FFmpeg is clearly better at fixing security issues.
>> To take a random example, an out of bounds read in the bink decoder was fixed
>> in FFmpeg three years ago [1], while Libav git master is still vulnerable
>> today.
>>
>> One can find more such examples, but I have better things to do with my time.
> 
> I think this attitude demonstrates a clear problem.

The clear problem that I see is that Libav upstream seems not to care very much
about fixing such things. This also discourages bug reporters.

> Instead of properly describing the issue through the appropriate channels,
> posts like this draw a bad picture by providing anectodal references of issues
> that may or may not affect Libav.

Anyone looking at the patch and the surrounding code (like where similar error
messages "Copy out of bounds" appear) can see that this problem does affect Libav.

> The right thing to do would be file a bugreport that collects references and
> information what the problem is and how to verify the fix, but this is not
> how this fight is fought apparently.

You yourself admitted that "the Libav bug tracking system is currently
dysfunctional" [a].
So why bother reporting problems, if they won't get fixed anyway?
I already mentioned Libav bug 840 [10], that has a sample and a command line crashing
Libav, but no reply since 6 weeks, while the same issue was fixed in FFmpeg very fast
(trac ticket #4431 [9]).

> But the issue is even more serious: Despite having the cleaner code-base, which
> compared to FFmpeg is much more accessible and easier to work with,

What do you mean with cleaner code-base?
I imagine it is a matter of taste.

> there are a significant number of very vocal developers that argue that FFmpeg was
> better in every thinkable way.

That's because this is mostly true. At least I haven't seen much evidence
contradicting it.

> Despite several calls for help, people have shown very little response such has
> pointing to specific patches and backporting/cherry picking missing functionality.
> There have been a handful of instances where this worked out though!

That's not very surprising, because that'd be duplicating work...

> In many many cases, the descriptions of the patches and the issues are sub-standard,
> in many cases even misleading. In no case that I looked at, the issue was immediately
> reproducible, because all of the referenced samples are held back and it is not easy
> at all the get access to them. And even if you do contact people via email and
> eventually are provided the samples, reproducing the issue remains very challenging.

Understanding the bink issue I mentioned above is not that challenging. Reproducing it
is a bit more difficult, but I can send you a sample if you can't create one yourself.
(However, I think that one doesn't need to have a sample to fix this issue, because
the problem is quite obvious and thus it really should have been fixed three years
ago in Libav as well.)

> I stopped looking actively at them when I repeatedly came to the conclusion that the
> issue can only be seen when seen when used in the test harnish that Google uses for
> testing libavcodec within chrome.

That might be the case for some (I don't know), but clearly not for all.
So not even looking at the patches is quite a bad idea.

>> > they (of course) also share their samples with both Michael (FFmpeg) and
>> > Luca/Anton (Libav). Michael seems to have much more capacity and time, and
>> > thus is usually faster with pushing patches for such crashers.
>>
>> Yes, Michael does an awesome job at fixing those [2].
> 
> He does an awesome job at producing patches that only a handful of people on this
> planet are capable of assessing what's going on. I cannot tell if he is obfuscating
> them on purpose, most likely this is rather the way he thinks and works on the issues:
> The work of a genius that takes a genius to understand.

How is the patch for the bink issue difficult to understand?

> To me, this constitutes a serious bus-factor: Without Michael, (probably) nobody is
> able to replace him.

Michael is the most active FFmpeg developer, but even without him FFmpeg would have
more manpower than Libav currently does (not counting that Libav changes get merged
into FFmpeg).
So even without Michael FFmpeg would probably do better than Libav does today.

> On the other hand, in such a scenario we wouldn't have the forks competing with
> each other either.

How can you know?

>> Interestingly Gentoo recently switched to FFmpeg by default [3] after conducting
>> a survey [4]. About 300 people participated in that survey and the outcome
>> was rather clear:
> 
> It saddens me to see people organizing votes where people participate that have
> no idea what they are voting about. How many of the 300 people that participated
> have tried to cherry pick a fix to libavcodec, or can even tell what the difference
> between libavcodec and libswscale is?

What does this matter if they encounter bugs with Libav, but not with FFmpeg?

>> > Libav on the other hand rather focuses on clean
>> > implementation and let's say better designed APIs.
>>
>> Let's say Libav focuses more on cosmetics like consistent indentation,
>> use of spaces around brackets, a linear commit history and such.
> 
> Please refrain from polemics such as these insults. Not everyone on this
> mailing list is as involved and familiar with both projects to clearly
> identify such statements as what they are.

I didn't mean this as insult, but rather as my personal observation,
similar to your personal observation above.
And I think this focus on cosmetics impedes Libav's development.
For example it took two weeks until my fix for checking the alac rice
limit [b] made it into Libav [c] and the only code change was cosmetics:
an added line break.

>> Less velocity in fixing (security) issues seems everything but appealing.
> 
> It is true that manpower is a scarce resource, in particular for volunteer-driven
> free software projects. In Debian, we are very aware of this problem and it makes
> my heart bleed to see this multimedia library and its users suffering from this fight.

This situation saddens me as well, but the least we can do is to provide Debian users
the better alternative.

> It appears to me that we in Debian have to choose. One option is to rely on Michael
> doing an awesome job and trusting that someone else will be able to pick up his work
> when it becomes necessary. Michael has been working on FFmpeg for a very very long time,
> but as we all know, manpower is the most precious resource available for free software
> projects and it is not realistic to drive the same project like forever.

Michael is by far not the only FFmpeg developer. You can for example look at statistics
about the past twelve month [d]:
                                Libav           FFmpeg
Contributors (Past 12 Months)   121 developers  306 developers
Commits (Past 12 Months)        1,756 commits   8,795 commits
Files Modified                  1,376 files     3,086 files 
Lines Added                     84,318 lines    299,504 lines
Lines Removed                   59,816 lines    184,442 lines 
Year-Over-Year Commits          Decreasing      Stable

> Libav was created because a number of developers disagreed with that option. Libavcodec
> and the related libararies constitute a high-profile and strategically relevant product
> with a large two-digit number of projects that rely on it. It really deserves
> well-understood code and APIs to ensure its usefulness for years and decades to come.

I agree that good APIs are important, but (security) bug fixes are at least as important.

>> I don't believe there would be more fragmentation than there is currently.
>> There could even be the contrary effect: If FFmpeg libraries are available
>> in Debian, there is less reason to embed a copy of them.
> 
> Well, we did have a precedent for this: Look at Debian in 2009-2010.
> It took very much work and a long time to get were we are today.
> It would be very sad to see this work undone.

I would be very sad as well, if this would get worse again. Therefore I have
no intention of letting it happen. :)
I even provided a patch for chromium to use the system FFmpeg libraries and
as far as I know this is currently the only application in Debian using an
embedded FFmpeg/Libav copy.

>> > I'm sure that this issue could be addressed in FFmpeg, I'm just not sure
>> > how insistent (and helpful) FFmpeg is here, because this for sure will
>> > compromise development velocity, which as written above, is the main
>> > disagreement between FFmpeg and Libav in my eyes. Maybe Andreas could
>> > comment on this point?
>>
>> How do you think FFmpeg could help in this regard?
> 
> I think less reliance on Michael would be extremely helpful. In the past,
> he has himself called for help several times on the ffmpeg mailing list,
> and putting even more work and responsibility on him adds to the risk.

What does this have to do with the fragmentation issue you talked about above?

I still don't know how you think FFmpeg can help preventing applications
from using different version as embedded copies.
What does Libav do against this?

>> My main reason for preferring to work on FFmpeg is that it already works better.
> 
> There are many use-cases where FFmpeg (in particular git master) constitutes
> the better product. It is easy to get the impression that this by itself makes
> it the better product for Debian and neglecting the additional constraints such
> as integration work and long-term maintainability risks.

I think FFmpeg requires less integration work, because more upstreams of applications
in Debian use it, and poses less long-term maintainability risks, because FFmpeg
upstream provides better long-term support.

>> > For Stretch, it is less clear. Libav seems to me like
>> > the prudent choice, FFmpeg like the more aggressive one.
>>
>> For stretch I think FFmpeg would be the sensible choice, while staying with
>> Libav would just be the 'status quo' choice.
> 
> Sensible for Debian?

Yes.

> I'm not sure. You bring a number of good points, but there is a significant long-term
> risk with relying on a single genius developer.

I doubt such a risk exists with FFmpeg.

> I'd rather love to see people working together than against each other,

Me too.

> but that might be wishful thinking.

One can get that impression if one looks at the history of the FFmpeg/Libav split...

> This is another way to explain why I'm so undecided what to recommend:
> On the one hand, I of course can follow the argument that technically,
> it appears easy to abandon libav and migrate Debian over to the FFmpeg
> variant and benefit from all the additional functionality that have
> accumulated over time.

Great!

> This adds additional significant risks.

I really don't see those significant risks.

The only arguments you mentioned is that FFmpeg has more decoders/demuxers and
thus a (slightly) larger attack surface and that Michael does a lot of work for
FFmpeg, so that it would be a big loss if he stopped working on it.
(Well, and you claim that Libav has a cleaner code base, but I don't know what
this is supposed to mean and neither do I understand why you think using
FFmpeg in Debian would risk increased usage of embedded copies.)

However, Libav doesn't even fix all known security issues in their smaller code
base in a timely manner. The result is that FFmpeg is better from a security
point of view.
Regarding the second argument, even if Michael vanished, the work he already did
would remain and a large part of that is not present in Libav. Additionally
there are many other FFmpeg developers. So this is no good argument against FFmpeg
either.

>> > I'd rather
>> > encourage people to help with porting missing features and bugs to Libav,
>> > because it is the cleaner and more maintainable codebase, but given that we
>> > are all volunteers, this may or may not happen.
>>
>> I don't think Libav has a much cleaner or more maintainable codebase, it's
>> just smaller.
> 
> Maybe you should just have a look,

I have looked at Libav code as well as FFmpeg code and they look quite similar
to me.

> it is really not that hard to port features
> over if they were implemented in a way that can be understood.

I think I already mentioned that I'm no fan of duplicated work like this.

>> Fixing issues in Libav that are already fixed in FFmpeg is
>> just duplicated work, which is not very rewarding to do.
> 
> It is much less rewarding to see features developed in Libav advertised as they
> originated in FFmpeg. But I guess this argument goes both ways.

These problems could be solved by "simply" reuniting...

> And neither way is of much relevance to Debian.

Indeed.

>> I'm not opposed to pkg-multimedia team maintenance for FFmpeg.
>> If anyone wants to help maintaining it, feel free to add yourself as uploader.
> 
> Thanks for the offer!

You're welcome.

Best regards,
Andreas


9: https://trac.ffmpeg.org/ticket/4431
10: https://bugzilla.libav.org/show_bug.cgi?id=840
a: https://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/044335.html
b: https://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=4b657a1b1eedcf38bcf36e89a2f4be6f76b5ce09
c: https://git.libav.org/?p=libav.git;a=commitdiff;h=243e8443cd9e83c887e3f5edf09a169e7783d14e
d: https://www.openhub.net/p/compare?project_0=libav&project_1=FFmpeg