Bug#948894: z3: add nojava build profile

[Helmut Grohne]

Hi Fabian,

On Tue, Jan 14, 2020 at 03:42:18PM +0100, Fabian Wolff wrote:
> [discussion continued from #948763]

Thank you!

> On 1/13/20 7:30 PM, Helmut Grohne wrote:
> > Why is the libz3-java package "Architecture: any" (long list actually)
> > instead of "Architecture: all"? Many lib*-java packages are
> > "Architecture: all" instead, so I'd like to understand the reason.
> > 
> > Why is the libz3-java package "Multi-Arch: foreign"? It seems to depend
> > on libz3-jni, which is "Multi-Arch: same". Such a setup seems wrong as
> > both seem to be libraries. As a dependency on libz3-java would be
> > expected to provide the jni components for the requested architecture,
> > but this is not the case here. Possibly, the "Multi-Arch: foreign" is
> > wrong here. In that case, "Architecture: any" does make sense as
> > "Architecture: all" cannot propagate an architecture-constraint. (This
> > is also known as the multi-arch-interpreter-problem.)
> > 
> > Why are libz3-java and libz3-jni separate packages? Why not merge them
> > into one? In which situations would you install one, but not the other?
> 
> First of all, I should note that I have adopted this package not too
> long ago from an inactive maintainer, so I'm still somewhat in the
> process of cleaning things up (and unfortunately, I can't ask the
> previous maintainer about decisions he's made because he won't reply
> to any emails).
> 
> From my understanding of what has happened, there used to be a single
> libz3-java package that included the shared library in usr/lib/*/jni/,
> which is why it had to be Architecture: any. It was also marked
> Multi-Arch: same, but because the JAR file in /usr/share/java/ was not
> built reproducibly, bug #797515 complained that the JAR file was an
> architecture-dependent file in a Multi-Arch: same package. It actually
> isn't architecture-dependent, of course, but because the build was
> (is?) not reproducible, rebuilding it on different architectures still
> yields different results (as does rebuilding on the same architecture).

Up to this point, this seems very plausible to me.

> The previous maintainer's solution to this was to split the libz3-java
> package into an Architecture: any, Multi-Arch: same libz3-jni package
> containing the JNI shared library and the current libz3-java package
> containing only arch-independent files, but because they did not build
> reproducibly, they *looked* like arch-dependent files, which is why, I
> suppose, he marked the package Architecture: any (instead of all) and
> Multi-Arch: foreign (because it's actually not arch-dependent).

Hmm. This sounds wicked indeed.

I checked the reproducibility issue using the backing database of the
multiarch hinter. The following query does not have any results.

SELECT * FROM contentconflict AS cc JOIN package AS p WHERE p.name IN
('libz3-jni', 'libz3-java') AND p.id = cc.pid1 AND p.id = cc.pid2;

As such, I think the reproducibility issue is now resolved.

> Of course, this solution is not very satisfying. Please correct me if
> I'm wrong, but I think that the proper solution should consist of a
> single libz3-java package marked Architecture: any (because of the JNI
> shared library) and Multi-Arch: same (for the JAR file), right?

I'm not entirely sure here. The lib*-jni vs lib*-java split seems common
in the java world. Though I also found some 20 packages stuffing shared
libraries onto lib*-java packages, so maybe that's allowed? Maybe ask
some java people for their policy here.

In any case, the JAR file is not the reason for "Multi-Arch: same". The
meaning of "Multi-Arch: same" is only that you can install the same
binary package for different architecture simultaneously on a system.
For this to work, all shared files (i.e. filenames present in multiple
instances) must have bit-identical content. Architecture-dependent paths
(e.g. anything below /usr/lib/<triplet>) can differ all the way they
want. It is also questionable whether we need Multi-Arch: same on java
stuff at all. I'm not sure there is a practical need at all. So marking
the java stuff "Multi-Arch: same" is a nice-to-have at best.

It seems that one way to simplify the situation could be merging the two
packages into one, providing the other and dropping the Multi-Arch stuff
if necessary (though it doesn't seem necessary).

What I can tell you is that having libz3-java depend on libz3-jni *and*
being marked Multi-Arch: foreign is almost certainly wrong. You should
either remove the foreign marking or the dependency. Likely, removing
the dependency is wrong.

> Apparently [0], JAR files still aren't built reproducibly, but
> dh_strip_nondeterminism might take care of this; do you know more
> about this? Comparing the amd64 and armhf builds of libz3-java from
> snapshot.debian.org actually does suggest that the package is built
> reproducibly by now.

Yes, the above sql query verified that it is presently reproducible
across all release architectures in unstable.

Now the next question is "how to add the nojava profile?". I can answer
that. The first step is adding "Build-Profiles: <!nojava>" to
debian/control for all java packages. And in principle, that's it. It
just happens to not be overly useful that way. The next step is making
code in debian/rules conditional to stop building java stuff. A quick
glimpse at the debian/rules suggests that this is already the case. :)
If you build with the nojava profile, dh_listpackages will not output
packages disabled by the profile. Thus WITH_JAVA will become OFF. The
last step is annotating the relevant Build-Depends with "<!nojava>".

There is a very easy way to check whether your profile is correctly
implemented: You do a full build and save the results. Then you do a
profile build (e.g. sbuild --profiles=nojava) and also save the results.
Then you run diffoscope on both of the .changes files. It should report
differences in the .buildinfo file and the .changes file, but not inside
the actual .debs. If all your .debs are reproducible, then with high
confidence your nojava profile is correctly implemented.

>From there, we can continue trying to cross build libz3 with nojava and
if that succeeds, try making the java stuff cross. The way we're
approaching it now splits the issue into manageable pieces.

Helmut