Are regression/unit test suites being run on upstream sources?

[Mark Wielaard]

Hi Barry,

On Sat, 2005-08-06 at 00:23 -0400, Barry Hawkins wrote:
>     The thought that people might be avoiding these test suites in order
> to ignore issues that surface in using the free runtimes is quite
> disconcerting to me, so I wanted to bring that up and just make sure
> that we are all on the same page that, at the very least, an upstream
> source should pass its own unit tests when built as a Debian package.
> Short-circuiting those tests is a favor to noone, and a disservice to
> our larger Debian community, whom we hope to introduce to the wealth of
> software that a solid Java packaging effort can provide.

Yes please! :)
And bug reports against unit tests are really appreciated (speaking for
kaffe/GNU Classpath/gcj) since they are often small and precise, which
makes analysis much easier. So if you disable a testsuite because it
fails with a free runtime, please at least file a bug report against it.

That said, I do have to admit that we as "upstream" are not yet doing
that great either. We are slowly trying to do a bit better, but help is
(as always) appreciated.

We have a large (30.000+ check points) core library test suite
(http://www.sourceware.org/mauve/) which is being extended continuously.
But it does contain tests for stuff that does fail (either because GNU
Classpath just doesn't have the required functionality yet or because a
certain runtime has some small/corner case bugs). So of around 30.000
checkpoints a few hundred still fail. A xfail file containing everything
that is a known bug/failure would be helpful here.

Secondly apache has begun starting to run gump for all their projects
using kaffe (http://gump.zones.apache.org/gump/kaffe/). This includes
running all the test suites. This is a great way to see how your package
is doing compared to "upstream". Unfortunately this installation is
being run on a solaris system and kaffe has some issues on that system
which result in occasional lockups there. We know that we can get a
score of at least 25% on a GNU/Linux setup. But even then we are still
not at 100% yet. If there is anybody that wants to create a Gump setup
on a Debian GNU/Linux (x86) using kaffe or gcj that would be a huge help
since testing and comparing results on such a setup is a lot easier for
most (upstream) hackers (since a lot do actually run Debian).

Cheers,

Mark
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.alioth.debian.org/pipermail/pkg-java-maintainers/attachments/20050806/ee9c227e/attachment.pgp