[Pkg-gmagick-im-team] lintian clean

[Daniel Kobras]

Hi!

On Mon, Jan 07, 2008 at 01:29:39PM -0200, Luciano Bello wrote:
> El Dom 06 Ene 2008, Nelson A. de Oliveira escribió:
> > Isn't libmagick-dev --> libmagick10-dev and libmagick++-dev -->
> > libmagick++10-dev?
> 
> Few mouth ago Daniel Kobras comments me:
> "I'd change the name of the -dev packages one more time in the next upload, 
> this time back to libmagick-dev and similar. This is the current 
> recommendation by the release managers because soname transitions can be 
> handled via bin-NMUs"
> 
> I couldn't found the link in lists.debian.org with the original 
> recommendation. Do you have it Daniel?

I don't have a link at hand, either - it's been communicated informally
several times on debian-release. I can give you the reasoning, though.
(I think Nelson recently mentioned that he wasn't all that familiar with
packaging libraries, so I'll try to elaborate a bit. Hopefully, it's useful.)

Libraries offer two kinds of interfaces, one at source level (API), and
a second one at the binary level (ABI) for the compiled executable.
Quite often, changes to one interface also affect the other one, but
there are cases where this doesn't hold true. For instance, consider a
renamed macro in a header file, eg. from

	#define True 1

to

	#define MagickTrue 1

that only affects the source level, whereas reordering the members of a
struct, eg from

	struct _Image {
		foo_t	foo;
		bar_t	bar;
	};

to

	struct _Image {
		bar_t	bar;
		foo_t	foo;
	};

only changes the binary interface (provided outside code directly
references struct members). Both examples are made up, but we've seen
similar changes to imagemagick more or less frequently in the past.
Especially changes of the second kind are applied every few releases for
no apparent reason.

To avoid breakage, a version identifier can be used to keep track of
incompatible changes. In a shared library, using such an identifier is
actually mandatory. It is called SONAME. When building a dynamically
linked binary, the (static) linker embeds information about the
required SONAMEs as into the executable. You can display them with eg.
'readelf -d /usr/bin/convert | grep NEEDED' or 'objdump -p
/usr/bin/mogrify | grep NEEDED'. For a given shared library, you can
query its embedded SONAME with 'readelf -d /usr/lib/libMagick.so.9 |
grep SONAME' or likewise using 'objdump -p'.

When a dynamically-linked binary is executed, the (dynamic) linker walks
its library search path and looks for file names that match SONAMEs
marked as NEEDED, and maps the library into the executable's address
space. Note: The linker does not check whether the SONAME requested in
NEEDED actually matches the SONAME embedded into the library, meaning
that you can play dirty tricks like 'ln -s libMagick.so.9
/usr/lib/libMagick.so.10'. The linker will now happily load
libMagick.so.9 whenever a program requests libMagick.so.10, even though
as with any other ABI breakage, it is unlikely to work correctly. The
ABI mismatch will usually cause segfaults or other bizarre breakage and
is usually a pain to debug.

During a transition period between two ABIs, some program are already
built with the new ABI, while others still require the old one.
Therefore, packages for the old and new version need to be
co-installable, meaning that they may not contain common files, and that
their package names need to differ. That's why we encode the SONAME into
the name of the package, eg. libMagick.so.9 corresponds to a package
libmagick9.

So much for shared libraries. Similar considerations also hold true for
the source-level interface, the API. The differences being, that there's
no standard or even mandatory method to version an API, and that its
quite common for applications to support multiple library APIs in
parallel, eg. by way of configure checks or some preprocessor magic. In
principle though, after an incompatible change to the API, we should
package the headers of the new version with a different name and install
to a different place without file overlap. If upstream did not change
location of the header files, we cannot avoid overlap, but we can still
change the package name and thus avoid having depending packages built
against the wrong API version. As ABI and API often change at the same
time, for simplicity we can re-use the SONAME as a version identifier
for the API as well. That is, if the SONAME tells us to build a package
libmagick10, we simply call the associated package containing the header
files libmagick10-dev, even though this need not be technically correct.

Indeed, that's what the library packaging guide at
http://www.netfort.gr.jp/~dancer/column/libpkg-guide/libpkg-guide.html#id291895
recommends, and it's the scheme I've introduced in the past for
imagemagick. It means that whenever imagemagick changes its SONAME,
maintainers of depending packages need to do a sourceful upload,
changing the Build-Depends line to the new version. They need to
acknowledge that their package still works with the new imagemagick, so
to speak.

This process is quite cumbersome, and experience tells that for most
libraries, it is not really necessary. Frequently, changes to the API
are minor and don't require any source changes for a majority of
applications. Nowadays, the release team can trigger automated rebuilds
(bin-NMUs) without bothering individual maintainers, which facilitates
library transitions to a new SONAME a lot. This requires that the -dev
package not change its name, however. Hence, the release team now
generally recommends a stationary name like libmagick-dev that doesn't
include a particular version. You should only change the name of
the -dev packages for a major API change, and if you want to keep
multiple, co-installable -dev packages in the archive in parallel.

Of course, even for a minor API change, there's the possibility that an
application fails to work with the new version. However, this usually
shows up as a build failure during the automated rebuilds, and doesn't
result in a broken package in the archive. As such, it can be spotted
quite easily. Uploads to the experimental suite can be used to
proactively find such problem cases before wrecking havoc in unstable.

The bottom line of all this for imagemagick is this: Imagemagick
regularly breaks its ABI, hence frequent library transitions are
necessary. From experience, most applications don't care, and these
transitions can be easily handled via bin-NMUs. To make everyone's lives
easier, we should keep a static name for the -dev package, either the
present libmagick9-dev, or do a final transition to the generic
name libmagick-dev as the former might cause some confusion once
libmagick9 vanishes.

By the way: In the past, imagemagick upstream hasn't paid close
attention to keeping binary compatibility between releases. So, if a new
upstream release didn't change its SONAME, it does not mean that the ABI
actually remained the same. Instead, its advisable to check the complete
diff between upstream versions and watch out for potential breakage.

Regards,

Daniel.