Bug#1111540: libfltk1.4: uses private implementation details of libdecor which might regress with new versions

[Simon McVittie]

Package: libfltk1.4
Version: 1.4.3-1
Severity: normal
Tags: upstream
X-Debbugs-Cc: libdecor-0 at packages.debian.org

libdecor has two sets of interfaces:

- a public API for use by libraries like SDL and FLTK, with symbols 
  declared in <libdecor.h>;
- a private interface for use by libdecor's Cairo and GTK plugins, with 
  symbols declared in "libdecor-plugin.h"

The public API is API- and ABI-stable, is available by including 
<libdecor.h> from libdecor-0-dev, and in the Debian packaging we've 
set it to generate dependencies like "libdecor-0-0 (>= 0.1.0)" in the 
usual way.

The plugin interface is not stable, and I believe upstream's position is 
that it may change at any time (like many projects they only support 
in-tree plugins, not out-of-tree plugins, to keep the size of their 
stable API under control). The key clue that it is not a stable 
interface is that libdecor-plugin.h is not installed. To ensure that 
libdecor plugins get the correct dependency relationship with libdecor, 
in the Debian packaging we've set the private symbols to generate 
lockstep dependencies like "libdecor-0-0 (= 0.2.3-1)" if used.

However, it turns out that FLTK does call some of the private functions, 
specifically libdecor_frame_get_content_height() and 
libdecor_frame_get_content_width() called from 
src/drivers/Wayland/Fl_Wayland_Window_Driver.cxx. It gets access to 
their declarations by having a vendored copy of libdecor, and using the 
libdecor-plugin.h from there. This is dangerous: the declarations in the 
vendored libdecor do not necessarily have anything to do with the system 
libdecor! For instance, if upstream changed the content width and height 
parameters from int to size_t for whatever reason, FLTK would be calling 
the function with the wrong ABI, leading to memory corruption.

If FLTK needs to call those two functions for whatever reason, the way 
to do this correctly would be to talk to libdecor upstream and arrange 
for them to be added to the stable public API in <libdecor.h> in a 
future release, and then depend on at least that version.

I think the signature of those two functions is unlikely to change in 
practice, so for now I'll set those two functions to generate ordinary 
(>=) dependencies and ask the release team to binNMU FLTK, so that 
libdecor can migrate to testing. However, before doing that, I wanted to 
have a bug report open to say that I consider this to be unsupported and 
a regression risk.

    smcv