[Pkg-javascript-devel] Bug#853035: node-liftoff: Non-determistically FTBFS due to unreliable timing in tests

[Chris Lamb]

Hi Graham,

> > I'm pretty hardcore on tests being determinstic, so merely increasing
> > the wait time feels especially gross to me. :)
> 
> Would you please explain your objection to timeouts in tests?

Well, we should probably separate the cases of using /timing/ in tests as
part of, say, checking whether something is "fast enough" or an algorithm is
O(n) vs O(n^2) and timing that is essentially relying on the machine not
being under load or is otherwise slow.

I am sure you would agree the former is problematic and not even remotely
determinstic! The latter is borderline, but when you can't rely on test
failures being genuine test failures, you psychologically start to demote
them in your mind in terms of importance and whether you should care about
the,.

> I think it is practical to give up waiting for a response some point.
> Even Debian's buildds give up after 150 minutes.

Mm, as a sanity check for the integrity of the test/build platform, timing
out after X hours or so makes sense and I have no problem with this. It's
just the stuff on the order of Y seconds that is a little bit more difficult
to justify. :)

(Can this test really not reliably detect whether something was killed/
spawned...?)


Best wishes,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby at debian.org / chris-lamb.co.uk
       `-