[PKG-Openstack-devel] Bug#834964: testresources: FTBFS too much often (failing tests)

[Santiago Vila]

[ I am more calmed now, thanks. I am going to disagree with you as
  friendly as I can in this message ].

BTW: If you missed this message, you should probably read it first:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=834964#20

On Sat, Sep 10, 2016 at 06:42:23PM +0200, Thomas Goirand wrote:
> On 09/09/2016 11:55 AM, Santiago Vila wrote:
> > The bug said stretch, and you tried sid, so you didn't really try to
> > reproduce it. Only because of this, you have already closed the bug
> > improperly, as you claim unreproducibility without actually testing it
> > the way it was reported.
> 
> Sid and Testing are supposed to be very close, and I didn't see anything
> special in the build dependencies that would make the package fail to
> build in Testing, but not in Sid. Do you have such a clue?

No, I don't have, but you never know for sure until you try it.

This is a matter of principles. Don't "interpret" bug reports.
Stick to the facts. The bug says stretch, you should try stretch,
especially if you are about to close the bug.

Ok, in this particular bug there is no difference, but that's not a
justification to try something different than what is being reported.

We could also argue that "dpkg-buildpackage" and "dpkg-buildpackage -A"
are very close, but there are a lot of packages which fail with -A
and do not fail without -A.

I have seen a case where the maintainer even claimed that using -A
"obviously" had no relation with the failure and refused to try the
build with -A. Well, the package did only fail with -A and it
was absolutely essential that it was tried that way to reproduce the
failure.

(In this case the error message was not the typical error message of a
buggy debian/rules, but that was no excuse for not trying -A).

> > Also, the bug said "FTBFS too much often". If you only tried to build
> > it once, you didn't try to reproduce it either, as you can't conclude
> > that the probability of something is zero just by trying it once.
> 
> Well, for other bugs you've reported, the issue was a missing
> build-dependency. Meaning that the package would *ALWAYS* fail. Anyone
> reasonable would then think that whenever you see an FTBFS, you're just
> always writing "FTBFS too much often" (by the way, shouldn't we just
> write "FTBFS too often" in correct English?).

Maybe. I don't know. I did the "google test" and believe this form to
be correct, maybe it's American English.
 
> I'd suggest that you make your bug report titles more accurate, then it
> will be less confusing.

This definitely deserves a clarification, as it seems there has been a
misunderstanding on your side about the subject of this bug.

Some people here have the theory (which I don't share) that it's ok
that a package FTBFS sometimes as far as it builds "most of the time",
i.e. the RC-ness is apparently based on a statistical measure.

My reaction to that is to file bugs with the subject that you have
already seen before this one: FTBFS too much often.

However, since a 100% failure rate is also "too much" (our goal is a
failure rate of 0%), the subject is also ok, by definition, for
packages which always fail.

Since I use a template and a script to create the bug reports, I don't
bother to remove the "too much often" from the subject when the
package always fail.

So it seems that you have interpreted this subject with the opposite
meaning. When I say "FTBFS too much often" I mean *exactly* that, too
much often, not "FTBFS always" (which would be stronger).

It may be 100%, it may be 50%, or it may be 8.2% as in this package.

So, as before: Don't interpret and stick to the facts.

If you doubt about the meaning of "too much often", just ask.

Closing the bug without asking is a little bit rude, IMHO.

> > And you claim that there is probably not a bug anywhere by closing the
> > report, despite evidence enough that there is a bug indeed?
> 
> Absolutely not.

Well, yes, that's the signal that you send when you close a bug.
"This is most probably not a real problem".

> I've closed the bug, and wrote that I was ok to investigate the
> issue with you.

Then you should not have closed the bug to begin with.

> Considering how you've done bug
> reporting for other issues (ie: your barbican & glance report where
> wrong, the issue was in pyopenssl, and you've insisted aggressively and
> rudely that we take the wrong actions of adding a version dependency
> when it was not needed), it was IMO normal to react this way.

No. The FTBFS bugs I reported for barbican and glance were real, not
imagined, so the reports were not "wrong" at all. Misdirected, maybe,
but not "wrong".

I'm a human like you and I make mistakes, but I try to be careful not
to report things (especially FTBFS bugs) which are not real bugs,
so you should not infer that something is "probably not a bug" just
because it's me who report it, because that's not only rude and
unprofessional, it's also wrong statistically speaking.

To summarize:

Consider each bug individually.
Stick to the facts.
Don't interpret.
Don't draw precipitate conclusions.

If we did that more often we would not have to discuss so much.

> I've reopened the bug and set severity to important, and it will stay
> with this severity until I can reproduce it myself.

Congratulations. I estimate that the failure rate is around 8.2% so
make sure that you try enough times for something with such
probability to happen.

But you will really go faster if you read the message I wrote
yesterday night:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=834964#20

After you read it, please consider raising to serious, it's a FTBFS bug.

Thanks.