[Pkg-emacsen-addons] Bug#818142: flycheck: FTBFS in testing (failing tests)

[Santiago Vila]

On Mon, 22 Aug 2016, Sean Whitton wrote:

> 1. flycheck tends to FTBFS because the test suite contains many tests
> which encode expected output from ~30 command line tools.  When one of
> those tools sees a new upstream version which tweaks its output, the
> test breaks.

Perhaps encoding the expected output (to the letter) is not a good
idea, then. Assuming those are standard Unix tools, they usually follow
some standard, which tells them how the output should be. A well
designed test should be prepared to check any output which follow this
standard, not the exact stream of bytes produced by a certain fixed
version.

> 2. The test suite also fails under certain autobuilder setups.  In
> particular, the reproducible builds team's automated builders fail with
> mysterious permission errors.  I think it is something to do with their
> tmpfs setup -- though I use tmpfs with sbuild and can't reproduce the
> error.

We are working with computers, which are designed to do that we tell
them to do. There are not really "mysterious errors", only errors
which nobody has investigated yet.

In particular, a build failure is usually one of two things: Either a
bug in the program (even if it does not always happen), or a bug in
the building software (I use sbuild, they use pbuilder).

So, I would encourage you to contact them to see if there are any bugs
in pbuilder (or their setup in general).

Thanks.