Select provider of libav* libraries

[Reinhard Tartler]

Please excuse my previous unfinished reply, it was sent in accident.
I'm not sure if this post really adds to this discussion, please consider
it as clarifications to my previous post.

On May 15, 2015 4:56 PM, "Andreas Cadhalpun"
> > I think security is not a decisive topic where either project cannot
> > clearly claim of having a clearly better handle.
>
> I disagree: FFmpeg is clearly better at fixing security issues.
> To take a random example, an out of bounds read in the bink decoder was
fixed
> in FFmpeg three years ago [1], while Libav git master is still vulnerable
> today.
>
> One can find more such examples, but I have better things to do with my
time.

I think this attitude demonstrates a clear problem. Instead of properly
describing the issue through the appropriate channels, posts like this draw
a bad picture by providing anectodal references of issues that may or may
not affect Libav. The right thing to do would be file a bugreport that
collects references and information what the problem is and how to verify
the fix, but this is not how this fight is fought apparently.

But the issue is even more serious: Despite having the cleaner code-base,
which compared to FFmpeg is much more accessible and easier to work with,
there are a significant number of very vocal developers that argue that
FFmpeg was better in every thinkable way. Despite several calls for help,
people have shown very little response such has pointing to specific
patches and backporting/cherry picking missing functionality. There have
been a handful of instances where this worked out though!

>
> > These days, FFmpeg for
> > sure asks for most (if not all) CVE numbers recently assigned, and
claims
> > to provide patches for them.
>
> FFmpeg not only claims to provide patches, but actually does provide them:
> most CVEs link to the corresponding patch.

In many many cases, the descriptions of the patches and the issues are
sub-standard, in many cases even misleading. In no case that I looked at,
the issue was immediately reproducible, because all of the referenced
samples are held back and it is not easy at all the get access to them. And
even if you do contact people via email and eventually are provided the
samples, reproducing the issue remains very challenging.

I stopped looking actively at them when I repeatedly came to the conclusion
that the issue can only be seen when seen when used in the test harnish
that Google uses for testing libavcodec within chrome.

>
> > they (of course) also share their samples with both Michael (FFmpeg) and
> > Luca/Anton (Libav). Michael seems to have much more capacity and time,
and
> > thus is usually faster with pushing patches for such crashers.
>
> Yes, Michael does an awesome job at fixing those [2].

He does an awesome job at producing patches that only a handful of people
on this planet are capable of assessing what's going on. I cannot tell if
he is obfuscating them on purpose, most likely this is rather the way he
thinks and works on the issues: The work of a genius that takes a genius to
understand.

To me, this constitutes a serious bus-factor: Without Michael, (probably)
nobody is able to replace him. On the other hand, in such a scenario we
wouldn't have the forks competing with each other either.

> Interestingly Gentoo recently switched to FFmpeg by default [3] after
conducting
> a survey [4]. About 300 people participated in that survey and the outcome
> was rather clear:

It saddens me to see people organizing votes where people participate that
have no idea what they are voting about. How many of the 300 people that
participated have tried to cherry pick a fix to libavcodec, or can even
tell what the difference between libavcodec and libswscale is?

[...]
> > I think the debate on the development methodology is the biggest
> > disagreement between the two projects: FFmpeg insists on keeping its
> > development velocity and allowing developers to "get work done",
> > compromising on maintainability and common understanding of the code
base
> > in favor of more features.
>
> I don't think getting work done is bad, because if work (like bug fixes)
> doesn't get done, it is worse for maintainability.
>
> > Libav on the other hand rather focuses on clean
> > implementation and let's say better designed APIs.
>
> Let's say Libav focuses more on cosmetics like consistent indentation,
> use of spaces around brackets, a linear commit history and such.

Please refrain from polemics such as these insults. Not everyone on this
mailing list is as involved and familiar with both projects to clearly
identify such statements as what they are.

> > This requires more work,
> > which in effect significantly lowers the development velocity. For a
system
> > integration job on the scale of Debian, less velocity seems more
appealing
> > to me.
>
> Less velocity in fixing (security) issues seems everything but appealing.
>

It is true that manpower is a scarce resource, in particular for
volunteer-driven free software projects. In Debian, we are very aware of
this problem and it makes my heart bleed to see this multimedia library and
its users suffering from this fight.

It appears to me that we in Debian have to choose. One option is to rely on
Michael doing an awesome job and trusting that someone else will be able to
pick up his work when it becomes necessary. Michael has been working on
FFmpeg for a very very long time, but as we all know, manpower is the most
precious resource available for free software projects and it is not
realistic to drive the same project like forever.

Libav was created because a number of developers disagreed with that
option. Libavcodec and the related libararies constitute a high-profile and
strategically relevant product with a large two-digit number of projects
that rely on it. It really deserves  well-understood code and APIs to
ensure its usefulness for years and decades to come.

> > Long-term support is not the only problem - I remain unconvinced that
> > switching from Libav to FFmpeg will solve the fragmentation problem. I
fear
> > that it would be replaced by fragmentation across different FFmpeg
upstream
> > versions. This may or may not be true in practice, but this is
> > what both my experience and intuition on this topic tells me.
>
> I don't believe there would be more fragmentation than there is currently.
> There could even be the contrary effect: If FFmpeg libraries are available
> in Debian, there is less reason to embed a copy of them.

Well, we did have a precedent for this: Look at Debian in 2009-2010. It
took very much work and a long time to get were we are today. It would be
very sad to see this work undone.

> > I'm sure that this issue could be addressed in FFmpeg, I'm just not sure
> > how insistent (and helpful) FFmpeg is here, because this for sure will
> > compromise development velocity, which as written above, is the main
> > disagreement between FFmpeg and Libav in my eyes. Maybe Andreas could
> > comment on this point?
>
> How do you think FFmpeg could help in this regard?

I think less reliance on Michael would be extremely helpful. In the past,
he has himself called for help several times on the ffmpeg mailing list,
and putting even more work and responsibility on him adds to the risk.

> > For me personally, this looks like a tie.
>
> For me it looks like a clear choice.
>
> > I would like to think that a
> > high-profile, high-performance and universal multimedia codec and format
> > library deserves a clean code-base, clean documentation, excellent test
> > coverage and a steady and thorough release process. Neither Libav nor
> > FFmpeg achieve that fully. Libav had so much potential, but struggles
with
> > achieving it for IMO mostly manpower reasons - working in or with FFmpeg
> > for some reason seems to be more attractive.
>
> My main reason for preferring to work on FFmpeg is that it already works
better.

There are many use-cases where FFmpeg (in particular git master)
constitutes the better product. It is easy to get the impression that this
by itself makes it the better product for Debian and neglecting the
additional constraints such as integration work and long-term
maintainability risks.

>
> > Clearly, Libav has lost a lot of its drive when it took off. This drive
has
> > significantly diminished - significant contributors from its inception
are
> > no longer active. I remain convinced that for the Jessie release, Libav
was
> > the better choice.
>
> Unfortunately FFmpeg was stuck in the NEW queue until after the
transition freeze,
> so for Jessie the choice was between Libav and Libav+FFmpeg, which the
security
> team didn't want.

By the time it was uploaded to NEW it was already unrealistic to conduct a
large-scale transition. It was about a year too late for that.

>
> > For Stretch, it is less clear. Libav seems to me like
> > the prudent choice, FFmpeg like the more aggressive one.
>
> For stretch I think FFmpeg would be the sensible choice, while staying
with
> Libav would just be the 'status quo' choice.

Sensible for Debian? I'm not sure. You bring a number of good points, but
there is a significant long-term risk with relying on a single genius
developer. I'd rather love to see people working together than against each
other, but that might be wishful thinking.

This is another way to explain why I'm so undecided what to recommend: On
the one hand, I of course can follow the argument that technically, it
appears easy to abandon libav and migrate Debian over to the FFmpeg variant
and benefit from all the additional functionality that have accumulated
over time. This adds additional significant risks. I can also see the
argument that compared to other risks that we take (see mysql/mariadb,
openoffice/libreoffice, etc.) this may or may not be acceptable.

> > I'd rather
> > encourage people to help with porting missing features and bugs to
Libav,
> > because it is the cleaner and more maintainable codebase, but given
that we
> > are all volunteers, this may or may not happen.
>
> I don't think Libav has a much cleaner or more maintainable codebase, it's
> just smaller.

Maybe you should just have a look, it is really not that hard to port
features over if they were implemented in a way that can be understood.

> Fixing issues in Libav that are already fixed in FFmpeg is
> just duplicated work, which is not very rewarding to do.

It is much less rewarding to see features developed in Libav advertised as
they originated in FFmpeg. But I guess this argument goes both ways. And
neither way is of much relevance to Debian.

> I'm not opposed to pkg-multimedia team maintenance for FFmpeg.
> If anyone wants to help maintaining it, feel free to add yourself as
uploader.

Thanks for the offer!

Reinhard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/attachments/20150518/84009118/attachment-0001.html>