Select provider of libav* libraries

Fri May 15 07:23:13 UTC 2015

Thanks for this insightful post, Dmitry,

On Mon, May 11, 2015 at 5:44 AM, Dmitry Smirnov <onlyjob at debian.org> wrote:
> On Sat, 9 May 2015 16:00:49 Andreas Cadhalpun wrote:
>> What would you count as very compelling reasons if more features, less
bugs
>> and better security support are not sufficient?
>
> More features is not necessary means less maintenance burden;
> Less bugs is not always means better software (it is a matter of how
upstream
> manages bugs);
> Quality of security support is something that remains to be seen.

I think security is not a decisive topic where either project cannot
clearly claim of having a clearly better handle. These days, FFmpeg for
sure asks for most (if not all) CVE numbers recently assigned, and claims
to provide patches for them. What is less visible is what happens behind
the scenes:

Most of these issues originate from Google Guys that work on security and
conduct fuzzing tests on libavcodec/libavformat. Their main target is of
course their own libavcodec/libavformat fork that they ship in Chrome, but
they (of course) also share their samples with both Michael (FFmpeg) and
Luca/Anton (Libav). Michael seems to have much more capacity and time, and
thus is usually faster with pushing patches for such crashers. Libav takes
the time to investigate, reproduce and understand those patches.
Unfortunately, in the majority of cases, this is not trivial at all, often
because of terse (or even wrong) commit messages, or the fact that there
are better places to fix a particular issue in the code. "Better" usually
means that more than a single instance of the issue is fixed.

Also, given that Libav supports significantly less codecs and formats (and
in some cases specific variants or features of codecs), many security
issues simply don't apply.

Libav could for sure do a much better job here by releasing faster. In the
past, I looked at doing new point releases about every 2-3 months, but for
family reasons, I wasn't able to keep that pace. Release frequency is
clearly something that needs improvement. As far as for the release
content, I am not aware of critical fixes being missed, so quality wise I
think we are good.

> I have a feeling that Debian already became life support for libav.
> Ever since Debian chosen libav, ffmpeg remained alive and apparently doing
> well without our help. I'm not too sure if libav would be able to stay
alive
> without Debian.

That's an interesting take on the matter. I don't think it is accurate; for
instance OpenEmbedded or Gentoo also ship Libav (in favor of FFmpeg), so
Debian&Ubuntu are clearly not alone. It is true that Libav owes the largest
portion of its userbase to Debian&Ubuntu, however even if Debian would stop
shipping Libav, then I doubt that this would threat Libav development in
any way.

>
> From maintenance prospective libav seems to be a liability. We have to
carry
> patches for packages where upstreams are not too concerned about
supporting
> libav.

This statement is a bit surprising to me. At the time Debian started to
ship Libav, there was absolutely *no* difference to FFmpeg. Since that,
many things have changed, and the two code-bases have diverged. While
FFmpeg started to merge as much functionality as possible, Libav focused on
maintainability, cleaned and straightened-up APIs and removed fringe codecs
and formats.

> Also in the light of past libav transitions and deprecations that required
> multiple changes in Debian and upstream I know no upstream who is happy to
> support libav.

In practice, maintainability means something different for an upstream
project and a distribution package. While upstream developers extend, fix
and refactor the code-base,as package maintainers we make sure that the
libavcodec package works fine with all applications that make use of it.

To be totally blunt: The libavcodec/libavformat API is horrible. This is
partly caused because of the aim to have a single library that covers every
thinkable container and codec flavor using a common API. Libav has
identified the development process as the culprit, where developers
submit unfinished and unreviewed work to the main codebase, which is then
used by applications and can basically never be fixed.

In an attempt to provide applications a better interface to
libavcodec/libavformat, Libav has in the past aggressively devised new
and deprecated old APIs for maintenance reasons, i.e., helping the
maintenance of the libav codebase.
For Debian with a large two digit number of reverse dependencies, this
caused a lot of fall-out that needed to be fixed. Libav developers were
instrumental with providing patches to make them build again. FFmpeg tries
hard to keep old APIs, which for sure lowers the effort for Linux
distributions, but is it really better to keep applications that use
obsolete and broken APIs? I'm not sure. In the long run, having
applications use less horrible APIs is for sure a good thing.

I think the debate on the development methodology is the biggest
disagreement between the two projects: FFmpeg insists on keeping its
development velocity and allowing developers to "get work done",
compromising on maintainability and common understanding of the code base
in favor of more features. Libav on the other hand rather focuses on clean
implementation and let's say better designed APIs. This requires more work,
which in effect significantly lowers the development velocity. For a system
integration job on the scale of Debian, less velocity seems more appealing
to me.

I'm having a very hard time to find an answer to the question which project
is better for Debian. One way of looking at it is which causes less effort
for the package maintainers. Andreas' and Dmity's mails make it sound that
Debian could save a lot of effort by switching to FFmpeg. I'm honestly not
sure if that is true. For instance, chromium upstream does not support
linking against a system libavcodec library. Andreas had to provide a
pretty invasive patch in #763632 for this, and I doubt that chromium
upstream is interested in merging it.

My biggest concern with the list that Andreas provided in
http://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/043980.html
is that the different upstream require specific upstream versions of
FFmpeg, and will break with other versions, which was pretty much the state
2009-2010 when I more or less took over of the package: Many upstream just
bundled some internal copy of libavcodec that they have developed against
and were updated with a very inconsistent pace. I'm happy to see that
FFmpeg finally no longer discourages the use of release tarballs in favor
for "svn trunk" or nowadays "git master" on the download page.
Nevertheless, the release frequency is just insanely fast, and long-term
support is practically non-existent: the oldest release branch was cut in
2014 -- for Debian/Ubuntu, we require significantly longer cycles.

Long-term support is not the only problem - I remain unconvinced that
switching from Libav to FFmpeg will solve the fragmentation problem. I fear
that it would be replaced by fragmentation across different FFmpeg upstream
versions. This may or may not be true in practice, but this is
what both my experience and intuition on this topic tells me.

I'm sure that this issue could be addressed in FFmpeg, I'm just not sure
how insistent (and helpful) FFmpeg is here, because this for sure will
compromise development velocity, which as written above, is the main
disagreement between FFmpeg and Libav in my eyes. Maybe Andreas could
comment on this point?

> I am not qualified for technical comparison between ffmpeg and libav.

I think nobody really is, because there is no technical metric or similar
criteria that could help us here.

What are the right questions for assessing FFmpeg vs Libav? Let me try to
formulate questions that focus on maintainability:

What project is less effort to package?
- I'd think Libav, because of the less frequent release cycle. Both
projects have rather accessible upstreams.

What project is doing a better job with handling defect reports?
- I'd think FFmpeg, because the Libav bug tracking system is currently
dysfunctional. AFAIUI this is being worked on. I've worked around this for
quite some time by talking to individuals on IRC directly, but this clearly
doesn't scale.

What project causes less effort for applications?
- A significant number of upstream prefer FFmpeg. OTOH, FFmpeg makes great
effort to provide source-code compatibility, but obviously a number of
projects only develop and test against FFmpeg, so this doesn't help.

What project is less effort for the security team?
- I'd say Libav, the security patches have significantly better commit
messages and descriptions, and generally make much more sense at least to
me. Maybe Moritz can elaborate on this.
My personal impression is that In FFmpeg, security support is handled by a
single person whose work is not exactly easy to follow.

For me personally, this looks like a tie. I would like to think that a
high-profile, high-performance and universal multimedia codec and format
library deserves a clean code-base, clean documentation, excellent test
coverage and a steady and thorough release process. Neither Libav nor
FFmpeg achieve that fully. Libav had so much potential, but struggles with
achieving it for IMO mostly manpower reasons - working in or with FFmpeg
for some reason seems to be more attractive.

Clearly, Libav has lost a lot of its drive when it took off. This drive has
significantly diminished - significant contributors from its inception are
no longer active. I remain convinced that for the Jessie release, Libav was
the better choice. For Stretch, it is less clear. Libav seems to me like
the prudent choice, FFmpeg like the more aggressive one. I'd rather
encourage people to help with porting missing features and bugs to Libav,
because it is the cleaner and more maintainable codebase, but given that we
are all volunteers, this may or may not happen.

Wow, that was a much longer email than I hoped. And given that I'm still
traveling and doing many family visits, it also took me much longer to
write this than I wished. If we really had to vote right now which way to
go, I'd probably abstain, because I also have to face the fact that I no
longer have the capacity to support a high-profile package like libavcodec
with the same time and devotion as I did for the last 5 years. How ever we
as a team decide to move forward, I think libavcodec (and related
libraries) are better team maintained (pkg-multimedia is just fine for
this), and I'd be happy to help out with the packaging either way.

-- 
regards,
Reinhard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/attachments/20150515/1bf9e26c/attachment.html>