Bug#734011: Can’t build a package using closure-compiler

[Colin Watson]

On Fri, Jan 03, 2014 at 09:49:09PM -0800, tony mancill wrote:
> On 01/03/2014 07:55 AM, David Prévot wrote:
> > Le 03/01/2014 11:34, tony mancill a écrit :
> >> On 01/02/2014 08:46 PM, Ben Finney wrote:
> >>> On 02-Jan-2014, David Prévot wrote:
> >>>> Calling closure-compiler inside pbuilder or sbuild (e.g. from d/rules)
> >>>> fails with:
> >>>>
> >>>> 	run-detectors: unable to find an interpreter for /usr/bin/closure-compiler
> > 
> >> […] I'll try to figure out
> >> what's going on with jarwrappers.  The only change with -4 upload is to
> >> depend on jarwrappers.
> > 
> > Haven’t double checked (so I don’t tag it directly with “found
> > 20130227+dfsg1-3”), but I’m pretty sure I ran into this bug with -3, and
> > only filled it against -4 after trying to understand it.
> 
> Hi David,
> 
> I'm seeing the same behavior with other packages that depend on
> jarwrapper - e.g. sat4j and logisim.
> 
> When I install them into a base sid chroot, they start correctly.  But
> when I install them into a chroot that has already has a JDK and a few
> other tools installed, the same error is emitted as you're seeing for
> closure-compiler:

The behaviour of binfmt_misc in conjunction with chroots is often a bit
hard to understand; although the kernel will execute the interpreter
in the same chroot context as the original executable, there's only one
/proc/sys/fs/binfmt_misc instance for the whole system right now, so the
interactions can be confusing.  You can largely get away with it as long
as things don't conflict ...

> > Setting up jarwrapper (0.45) ...
> > update-binfmts: warning: jarwrapper already enabled in kernel.

This implies that jarwrapper was already installed in the base system.

> Attempting the same in the non-base sid chroot (still stock sid, but
> just created a while back with the JDK already installed), the
> jarwrapper postinst output is different:
> 
> > (jdksid-chroot) apt-get install jarwrapper
> > [...]
> > The following extra packages will be installed:
> >   binfmt-support fastjar init-system-helpers
> > The following NEW packages will be installed:
> >   binfmt-support fastjar init-system-helpers jarwrapper
> > [...]
> > Setting up binfmt-support (2.1.1-1) ...
> > update-binfmts: warning: python3.3 already enabled in kernel.
> > update-binfmts: warning: found manually created entry for python2.7 in /proc/sys/fs/binfmt_misc; leaving it alone
> > update-binfmts: warning: found manually created entry for jar in /proc/sys/fs/binfmt_misc; leaving it alone
> > update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
> > invoke-rc.d: policy-rc.d denied execution of start.
> > Setting up fastjar (2:0.98-5) ...
> > Setting up jarwrapper (0.45) ...
> > update-binfmts: warning: found manually created entry for jarwrapper in /proc/sys/fs/binfmt_misc; leaving it alone

This is basically the same underlying error at a slightly different
point.

> And no entry is created for jarwrapper, so the magic number for jars
> isn't known to run-detectors.

Right, hmm.  Possibly we're bailing out a bit early.  I think I see the
problem, but this is non-trivial enough that I'd like to track it
properly; could you please file a bug against binfmt-support for me, and
attach tarballs of /usr/share/binfmts/ and /var/lib/binfmts/ from the
various chroots, plus a tarball of /proc/sys/fs/binfmt_misc/ (will be
the same in all filesystem contexts)?

> It doesn't appear to be related to the most recent update to
> binfmt-support because I see the same behavior in jessie as on sid if I
> install default-jre *before* installing jarwrapper and binfmt-support.
> Put another way, it seems to boil down to whether binfmt-support and
> jarwrapper are able to be configured before the JRE/JDK/java-common
> packages.
> 
> If jarwrapper is installed first, we have this situation:
> 
> $ update-binfmts --display jarwrapper
> jarwrapper (enabled):
>      package = <local>
>         type = magic
>       offset = 0
>        magic = PK\x03\x04
>         mask =
>  interpreter = /usr/bin/jarwrapper
>     detector = /usr/bin/jardetector
> 
> But the "jar" binfmt is unhappy:
> 
> $ update-binfmts --display jar
> update-binfmts: warning: jar not in database of installed binary formats.
> update-binfmts: exiting due to previous errors
> 
> Reverse the order of package installation, and you get this:
> 
> $ update-binfmts --display jarwrapper
> update-binfmts: warning: jarwrapper not in database of installed binary
> formats.
> update-binfmts: exiting due to previous errors
> 
> $ update-binfmts --display jar
> jar (enabled):
>      package = openjdk-7
>         type = magic
>       offset = 0
>        magic = PK\x03\x04
>         mask =
>  interpreter = /usr/bin/jexec
>     detector =

The error messages you cite above relate to whether the binfmt *name* is
already registered with binfmt_misc, which shouldn't have anything to do
with whether they use the same magic number or not.  The coincidence is
a bit odd, though, and I'll probably need to try it out ...

> In any event, I feel pretty confident that this bug doesn't belong to
> libclosure-compiler-java, but instead with jarwrappers and/or openjdk-7,
> although it's going to take some more digging to figure out what to do
> about two packages trying to register the same magic number with
> binfmt-support.  (Colin, I hope that you don't mind that I added you to
> the cc:  I thought you could probably say right off whether its feasible
> to have two packages trying to use binfmt-support in this manner.)

This is what I designed the "detectors" scheme for: you can find
documentation of it in doc/detectors in the binfmt-support source
package (or /usr/share/doc/binfmt-support/detectors as of 2.1.1-1).  If
a set of packages need to install binfmts with the same magic or the
same extension, then they're ideally supposed to add detectors to their
binfmt specification to further narrow down which real interpreter ought
to be executed.

In this case, what actually happens is that run-detectors will try
anything with a detector first, followed by anything without a detector;
so jardetector will get its turn first, and then it will fall back to
jexec if jardetector declines.  I guess that's OK?  We do need to figure
out why update-binfmts isn't registering things correctly, though.  I
notice that /usr/share/binfmts/jar is an alternative, which may have
some interesting effects; compare #734197.

-- 
Colin Watson                                       [cjwatson at debian.org]