Select provider of libav* libraries

Sat May 16 09:36:49 UTC 2015

Hi Reinhard,

2015-05-15 9:23 GMT+02:00 Reinhard Tartler <siretart at gmail.com>:
> Thanks for this insightful post, Dmitry,
>
> On Mon, May 11, 2015 at 5:44 AM, Dmitry Smirnov <onlyjob at debian.org> wrote:
>> On Sat, 9 May 2015 16:00:49 Andreas Cadhalpun wrote:
>>> What would you count as very compelling reasons if more features, less
>>> bugs
>>> and better security support are not sufficient?
>>
>> More features is not necessary means less maintenance burden;
>> Less bugs is not always means better software (it is a matter of how
>> upstream
>> manages bugs);
>> Quality of security support is something that remains to be seen.
>
> I think security is not a decisive topic where either project cannot clearly
> claim of having a clearly better handle. These days, FFmpeg for sure asks
> for most (if not all) CVE numbers recently assigned, and claims to provide
> patches for them. What is less visible is what happens behind the scenes:
>
> Most of these issues originate from Google Guys that work on security and
> conduct fuzzing tests on libavcodec/libavformat. Their main target is of
> course their own libavcodec/libavformat fork that they ship in Chrome, but
> they (of course) also share their samples with both Michael (FFmpeg) and
> Luca/Anton (Libav). Michael seems to have much more capacity and time, and
> thus is usually faster with pushing patches for such crashers. Libav takes
> the time to investigate, reproduce and understand those patches.
> Unfortunately, in the majority of cases, this is not trivial at all, often
> because of terse (or even wrong) commit messages, or the fact that there are
> better places to fix a particular issue in the code. "Better" usually means
> that more than a single instance of the issue is fixed.
>
> Also, given that Libav supports significantly less codecs and formats (and
> in some cases specific variants or features of codecs), many security issues
> simply don't apply.
>
> Libav could for sure do a much better job here by releasing faster. In the
> past, I looked at doing new point releases about every 2-3 months, but for
> family reasons, I wasn't able to keep that pace. Release frequency is
> clearly something that needs improvement. As far as for the release content,
> I am not aware of critical fixes being missed, so quality wise I think we
> are good.
>
>> I have a feeling that Debian already became life support for libav.
>> Ever since Debian chosen libav, ffmpeg remained alive and apparently doing
>> well without our help. I'm not too sure if libav would be able to stay
>> alive
>> without Debian.
>
> That's an interesting take on the matter. I don't think it is accurate; for
> instance OpenEmbedded or Gentoo also ship Libav (in favor of FFmpeg), so
> Debian&Ubuntu are clearly not alone. It is true that Libav owes the largest
> portion of its userbase to Debian&Ubuntu, however even if Debian would stop
> shipping Libav, then I doubt that this would threat Libav development in any
> way.
>
>>
>> From maintenance prospective libav seems to be a liability. We have to
>> carry
>> patches for packages where upstreams are not too concerned about
>> supporting
>> libav.
>
> This statement is a bit surprising to me. At the time Debian started to ship
> Libav, there was absolutely *no* difference to FFmpeg. Since that, many
> things have changed, and the two code-bases have diverged. While FFmpeg
> started to merge as much functionality as possible, Libav focused on
> maintainability, cleaned and straightened-up APIs and removed fringe codecs
> and formats.
>
>> Also in the light of past libav transitions and deprecations that required
>> multiple changes in Debian and upstream I know no upstream who is happy to
>> support libav.
>
> In practice, maintainability means something different for an upstream
> project and a distribution package. While upstream developers extend, fix
> and refactor the code-base,as package maintainers we make sure that the
> libavcodec package works fine with all applications that make use of it.
>
> To be totally blunt: The libavcodec/libavformat API is horrible. This is
> partly caused because of the aim to have a single library that covers every
> thinkable container and codec flavor using a common API. Libav has
> identified the development process as the culprit, where developers
> submit unfinished and unreviewed work to the main codebase, which is then
> used by applications and can basically never be fixed.
>
> In an attempt to provide applications a better interface to
> libavcodec/libavformat, Libav has in the past aggressively devised new
> and deprecated old APIs for maintenance reasons, i.e., helping the
> maintenance of the libav codebase.
> For Debian with a large two digit number of reverse dependencies, this
> caused a lot of fall-out that needed to be fixed. Libav developers were
> instrumental with providing patches to make them build again. FFmpeg tries
> hard to keep old APIs, which for sure lowers the effort for Linux
> distributions, but is it really better to keep applications that use
> obsolete and broken APIs? I'm not sure. In the long run, having applications
> use less horrible APIs is for sure a good thing.
>
> I think the debate on the development methodology is the biggest
> disagreement between the two projects: FFmpeg insists on keeping its
> development velocity and allowing developers to "get work done",
> compromising on maintainability and common understanding of the code base in
> favor of more features. Libav on the other hand rather focuses on clean
> implementation and let's say better designed APIs. This requires more work,
> which in effect significantly lowers the development velocity. For a system
> integration job on the scale of Debian, less velocity seems more appealing
> to me.
>
> I'm having a very hard time to find an answer to the question which project
> is better for Debian. One way of looking at it is which causes less effort
> for the package maintainers. Andreas' and Dmity's mails make it sound that
> Debian could save a lot of effort by switching to FFmpeg. I'm honestly not
> sure if that is true. For instance, chromium upstream does not support
> linking against a system libavcodec library. Andreas had to provide a pretty
> invasive patch in #763632 for this, and I doubt that chromium upstream is
> interested in merging it.
>
> My biggest concern with the list that Andreas provided in
> http://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/043980.html
> is that the different upstream require specific upstream versions of FFmpeg,
> and will break with other versions, which was pretty much the state
> 2009-2010 when I more or less took over of the package: Many upstream just
> bundled some internal copy of libavcodec that they have developed against
> and were updated with a very inconsistent pace. I'm happy to see that FFmpeg
> finally no longer discourages the use of release tarballs in favor for "svn
> trunk" or nowadays "git master" on the download page. Nevertheless, the
> release frequency is just insanely fast, and long-term support is
> practically non-existent: the oldest release branch was cut in 2014 -- for
> Debian/Ubuntu, we require significantly longer cycles.
>
> Long-term support is not the only problem - I remain unconvinced that
> switching from Libav to FFmpeg will solve the fragmentation problem. I fear
> that it would be replaced by fragmentation across different FFmpeg upstream
> versions. This may or may not be true in practice, but this is
> what both my experience and intuition on this topic tells me.
>
> I'm sure that this issue could be addressed in FFmpeg, I'm just not sure how
> insistent (and helpful) FFmpeg is here, because this for sure will
> compromise development velocity, which as written above, is the main
> disagreement between FFmpeg and Libav in my eyes. Maybe Andreas could
> comment on this point?
>
>> I am not qualified for technical comparison between ffmpeg and libav.
>
> I think nobody really is, because there is no technical metric or similar
> criteria that could help us here.
>
> What are the right questions for assessing FFmpeg vs Libav? Let me try to
> formulate questions that focus on maintainability:
>
> What project is less effort to package?
> - I'd think Libav, because of the less frequent release cycle. Both projects
> have rather accessible upstreams.
>
> What project is doing a better job with handling defect reports?
> - I'd think FFmpeg, because the Libav bug tracking system is currently
> dysfunctional. AFAIUI this is being worked on. I've worked around this for
> quite some time by talking to individuals on IRC directly, but this clearly
> doesn't scale.
>
> What project causes less effort for applications?
> - A significant number of upstream prefer FFmpeg. OTOH, FFmpeg makes great
> effort to provide source-code compatibility, but obviously a number of
> projects only develop and test against FFmpeg, so this doesn't help.
>
> What project is less effort for the security team?
> - I'd say Libav, the security patches have significantly better commit
> messages and descriptions, and generally make much more sense at least to
> me. Maybe Moritz can elaborate on this.
> My personal impression is that In FFmpeg, security support is handled by a
> single person whose work is not exactly easy to follow.
>
> For me personally, this looks like a tie. I would like to think that a
> high-profile, high-performance and universal multimedia codec and format
> library deserves a clean code-base, clean documentation, excellent test
> coverage and a steady and thorough release process. Neither Libav nor FFmpeg
> achieve that fully. Libav had so much potential, but struggles with
> achieving it for IMO mostly manpower reasons - working in or with FFmpeg for
> some reason seems to be more attractive.
>
> Clearly, Libav has lost a lot of its drive when it took off. This drive has
> significantly diminished - significant contributors from its inception are
> no longer active. I remain convinced that for the Jessie release, Libav was
> the better choice. For Stretch, it is less clear. Libav seems to me like the
> prudent choice, FFmpeg like the more aggressive one. I'd rather encourage
> people to help with porting missing features and bugs to Libav, because it
> is the cleaner and more maintainable codebase, but given that we are all
> volunteers, this may or may not happen.
>
> Wow, that was a much longer email than I hoped. And given that I'm still
> traveling and doing many family visits, it also took me much longer to write
> this than I wished. If we really had to vote right now which way to go, I'd
> probably abstain, because I also have to face the fact that I no longer have
> the capacity to support a high-profile package like libavcodec with the same
> time and devotion as I did for the last 5 years. How ever we as a team
> decide to move forward, I think libavcodec (and related libraries) are
> better team maintained (pkg-multimedia is just fine for this), and I'd be
> happy to help out with the packaging either way.
Thank you for sharing your opinion and vote.

We now have 7 votes for FFmpeg:
Bálint [1], Alessandro [2], Fabian [3]  Andreas [4], IOhannes [5],
Dmitry [6] and Fabian [7].

Two votes for Libav:
Jonas [8] and Alessio [9].

One developer abstains:
Reinhard [10].

Alessio's email [9] also signaled that his position may be different but
he voted for Libav to break consensus till Reinhard's opinion arrives:
> Alright, let me fix this then by changing my mind on the subject (and
> giving us more time to debate):
>
>    I vote for libav to stay until very compelling reasons to switch
>    to ffmpeg are provided.
>
> There is no room for statements like "clear consensus" now - by
definition.

Alessio, if you would like to change your vote this may be a good time
for that.

If I misinterpreted any position or you would like to change your vote,
please update the poll results.

Andreas also pointed out that in a poll [11] among Gentoo users ffmpeg
won over libav with a huge margin.

--

I still believe that FFmpeg would be a significantly better choice for
Debian as default and Andreas' email [12] covers most of the things I
wanted to answer to this email thus I don't repeat those there.

IMO the poll at Gentoo should play more a important role in the decision
than the poll conducting among ourselves since we should focus on our
users' needs rather than our personal preferences.

I can't recommend Debian anyone using this line:
"It can't play your videos but it has a nice API."

Cheers,
Balint

1:
https://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-April/043930.html
2:
https://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-April/043939.html
3:
https://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/044078.html
4:
https://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/043980.html
5:
https://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/044180.html
6:
https://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/044234.html
7:
https://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/044236.html
8:
https://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/044341.html
9:
https://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/044184.html
10:
https://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/044335.html
11: https://forums.gentoo.org/viewtopic-t-1010096.html
12:
https://lists.alioth.debian.org/pipermail/pkg-multimedia-maintainers/2015-May/044346.html