Bug#749944: build ogre-1.9 against new boost-defaults 1.55?

Sat May 31 09:34:33 UTC 2014

2014-05-31 2:06 GMT+01:00 Scott Howard <showard314 at gmail.com>:
> reopen 749944
>
> On Fri, May 30, 2014 at 7:50 PM, Manuel A. Fernandez Montecelo
> <manuel.montezelo at gmail.com> wrote:
>> 2014-05-31 0:18 GMT+01:00 Scott Howard <showard314 at gmail.com>:
>>> On Fri, May 30, 2014 at 4:46 PM, Manuel A. Fernandez Montecelo
>>> <manuel.montezelo at gmail.com> wrote:
>>>> The versions are hardcoded to force reverse build-depends using the
>>>> same versions, due to problems in the past (incompatibilities causing
>>>> deadlocks when starting the application):
>>>
>>> Thank you for explaining. I'll update my package to follow ogre's
>>> hardcoded version of boost manually to prevent that bug from
>>> happening.
>>
>> Is that MyGui?
>
> mygui and openmw.
>
> I'm CC:ing Bret to this email, he's been maintaining them.
>
> Bret: Ogre exposes the boost ABI, so to prevent situations where
> previously compiled programs break when boost is bumped, ogre-1.9 is
> hardcoded to a specefic library version.
>
> Manuel, what do you think of this (please correct me if I'm wrong):
> Since packages depending on ogre-1.9 are built against other libraries
> that use boost-defaults, that causes them to FTBFS when the boost
> transition occurs. Ogre-1.9 does not transition with the rest of the
> archive. Packages really should depend on the version of boost in
> boost-defaults otherwise the archive might become inconsistent (you
> can see that 350 packages in the archive depend on boost, and ogre is
> one one of 4 that do not depend on the default version defined by
> boost-defaults). To keep the archive consistent and support the user
> case defined in bug 674633 I propose
> 1) ogre-1.9 depend on libboost-dev
> 2) libogre-1.9-dev depend on: libboost-dev (which would pull in the
> proper version and conflict for those users that are developing with
> another library, addressing bug 674633).
>
>
> Also:
> "But it takes a bit of time, disk space (several GBs) of which I don't
> have much at the moment, and probably binNMUs to reverse dependencies
> just in case that something like that bug happens again (ogre-1.9
> compiled with 1.55 and rdeps having been compiled with 1.54 from the
> previous versions forced by ogre-1.9 could cause a problem similar to
> the previous bug)."
>
> That is what is supposed to happen when the whole archive transitions
> to a new version of boost. If ogre had a BD on boost-defaults
> (libboost*-dev), ogre and ogre's dependencies would have been binNMUed
> by the team performing the transition. Instead, the whole archive
> transitions, except ogre. Ogre's rdepends that also depend on either
> libboost-default or any other library that depends on libboost-default
> FTBFS and have to be removed from testing in order for the boost
> transition to complete. This isn't very important now, since few
> packages depend on ogre-1.9, but as the number of packages depending
> on ogre-1.9 increase, it will become increasingly important to depend
> on boost-defaults (libboost-dev).
>
> The bug described in 674633 is unfortunate, it will never occur in a
> stable release (were boost versions are not changing). I believe the
> emphasis should be on maintaining archive consistency which will mean
> users using unstable/testing as a development environment will break
> on occasion. You can reduce the effects of that breakage by
> build-depending on libboost-dev, thus forcing those users to also
> upgrade their code/compilation to the new versions of boost as well.

I am a bit busy this weekend so I don't have time to relearn all of
the intricacies of the issue, but from memory I think that it didn't
happen exactly as you describe or with the same consequences.

Just for reference, one thing that people has to have in mind before
reading that bug report is CEGUI was not the Debian package but an
upstream version (so it was not only an internal problem of Debian and
transitions of packages in the archive), and the reporter was the lead
developer of CEGUI.

I also don't know if CEGUI was using boost::thread directly or only
through OGRE (if it was using boost at the time it was for other
libraries and not for the same thread thing; I think), and possibly
using a way of compilation that introduced the problem directly into
the binary (static compilation), not calling the dynamic library.

With the next version of boost, OGRE was binNMUed or equivalent, and
the code from boost was updated in a way that was incompatible with
the previous code which was present in CEGUI binary (either for legit
reasons or by mistakes in the libraries or their usage).

Say, with the old version, CEGUI got statically compiled a version
doing something like (probably nothing like what really happened, but
hopefully helps explain the problem):

lock(resrcmgr_lock)
lock(resrc_a)
do_stuff()
lock(resrc_b)

With the new version, OGRE had something like:

lock(resrc_a)
lock(resrcmgr_lock)
do_stuff()
lock(resrc_b)

For some reason, things interlocked in initialisation every time.

In that case, since the code of CEGUI is updated only at compile time,
it cannot be fixed without a recompilation.  Since at the time we were
not forcing a dependency of a specific version in libogre-dev (or not
even a dependency at all, cannot remember), even when recompiling the
problem was not fixed, because the reporter was compiling against an
incompatible version of Boost (1.48) while OGRE was using 1.49.  (I
don't know if nowadays they can be installed side by side, at that
point it seems that it was possible).

If we now add depends in libogre-dev on libboost-thread-dev, and this
one in turn specifies the version to be libboost-thread1.55-dev, this
probably works for packages installed in the system *if they are
recompiled*.

But it will not work for the CEGUI of the archive, since the binary
packages don't have dependencies on libboost-thread, so they would not
be NMUed when Boost is bumped, nor when OGRE gets binNMUed because of
that.  So it would continue with the possibly problematic static code
compiled, possibly incompatible with the never version that binNMUed
OGRE uses after bumping Boost versions.

I don't know how likely is that the problems appear again, maybe it
was a once-in-a-lifetime change of Boost to unique_lock that happens
again, but I think that the problem can happen and indeed, happened in
the past.

It can also happen in stable releases, if CEGUI (or other rev-deps)
were compiled for the last time 6 months before the freeze, and a new
Boost is set as default 2 months before the freeze and OGRE binNMUd.

By forcing a specific version OGRE, it will not have this silent
breaking of CEGUI (or other reverse-depends), since the code that can
cause problems in Debian's OGRE and CEGUI/other-rdeps are upgraded in
lockstep (as long as I remember to tell them or ask for binNMUs), and
also in lockstep rev-depends on OGRE after they are recompiled.

I acknowledge that this situation with not using boost-defaults is a
problem, and forces packages in Debian depending on OGRE using Boost
themselves to use the same versions, which is inconvenient and has
ripple effects.

But I just don't know how to get out of it without potentially causing
those problems again.

Does this sound a sensible explanation?  Any ideas?

(BTW, ogre-1.9 depends on 1.54 but ogre-1.8, which I hoped that was
removed by now, has generic dependency on libboost*-dev as you
described above).

Cheers.
-- 
Manuel A. Fernandez Montecelo <manuel.montezelo at gmail.com>