[DRE-maint] Bug#795676: ruby-text-table: Non-deterministic FTBFS due to unreliable timing benchmarks in tests

[Chris Lamb]

Source: ruby-text-table
Version: 1.2.3-2
Severity: serious
Justification: fails to build from source

Dear Maintainer,

ruby-text-table can non-deterministic FTBFS due to using unreliable
timing benchmarks in tests:

  [..]
  
    1) Text::Table performance is linear relative to row count
       Failure/Error: time.should_not > base * 12
         expected not: > 0.015996084
                  got:   0.022761029
       # ./spec/integration/performance_spec.rb:9:in `block (2 levels)
       in <top (required)>'
  
  [..]

The code itself (spec/integration/performance_spec.rb):

  require 'spec_helper'
  require 'benchmark'
  
  describe Text::Table, 'performance' do
    it 'is linear relative to row count' do
      base = time_to_render_num_of_rows 30
      time = time_to_render_num_of_rows 300
  
      time.should_not > base * 12
    end
  
    def time_to_render_num_of_rows(num)
      GC.start
  
      Benchmark.realtime do
        Text::Table.new(:rows => Array.new(num)).to_s
      end
    end
  end

What can happen is that if ``time_to_render_num_of_rows 300`` takes a
long time to run (eg. sudden high load, suspend, etc. etc.), the test
incorrectly concludes that the algorithm is not linear and the build
fails.

The test is completely unreliable anyway -- if
``time_to_render_num_of_rows 30`` runs relatively /fast/ for some reason
(CPU throttling disabled, whatever) and the algorithm is polynomial, the
test will incorrectly *pass*. So, basically, it's garbage.

The full build log is attached or can be viewed here:

  https://reproducible.debian.net/logs/unstable/amd64/ruby-text-table_1.2.3-2.build1.log.gz


Regards,

-- 
      ,''`.
     : :'  :     Chris Lamb
     `. `'`      lamby at debian.org / chris-lamb.co.uk
       `-
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: ruby-text-table.1.2.3-2.unstable.amd64.log.txt
URL: <http://lists.alioth.debian.org/pipermail/pkg-ruby-extras-maintainers/attachments/20150816/13ca8f8f/attachment-0001.txt>