[DRE-maint] Bug#1008214: ruby-simplecov: autopkgtest regression: error output changed

Paul Gevers elbrus at debian.org
Thu Mar 24 12:25:20 GMT 2022 

Source: ruby-simplecov
Version: 0.21.2-1
Severity: serious
User: debian-ci at lists.debian.org
Usertags: regression

Dear maintainer(s),

With a recent upload of ruby-simplecov the autopkgtest of ruby-simplecov 
fails in testing when that autopkgtest is run with the binary packages 
of ruby-simplecov from unstable. It passes when run with only packages 
from testing. In tabular form:

                        pass            fail
ruby-simplecov         from testing    0.21.2-1
all others             from testing    from testing

I copied some of the output at the bottom of this report.

Currently this regression is blocking the migration to testing [1]. Can 
you please investigate the situation and fix it?

More information about this bug and the reason for filing it can be found on
https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation

Paul

[1] https://qa.debian.org/excuses.php?package=ruby-simplecov

https://ci.debian.net/data/autopkgtest/testing/amd64/r/ruby-simplecov/20267102/log.gz


Failures:

   1) return codes inside fixtures/frameworks when running 
testunit_bad.rb behaves like bad tests with default configuration prints 
a message to STDERR
      Failure/Error: expect(@stderr).to 
match(/stopped.+SimpleCov.+previous.+error/i)

        expected 
"<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in 
`require': cannot 
loa...r_ruby/rubygems/core_ext/kernel_require.rb>:85:in 
`require'\n\tfrom testunit_bad.rb:2:in `<main>'\n" to match 
/stopped.+SimpleCov.+previous.+error/i
        Diff:
        @@ -1,3 +1,5 @@
        -/stopped.+SimpleCov.+previous.+error/i
 
+<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in 
`require': cannot load such file -- lib/simplecov (LoadError)
        +	from 
<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in 
`require'
        +	from testunit_bad.rb:2:in `<main>'
      Shared Example Group: "bad tests" called from 
./spec/return_codes_spec.rb:66
      # ./spec/return_codes_spec.rb:37:in `block (5 levels) in <top 
(required)>'
      # ./spec/return_codes_spec.rb:12:in `block (4 levels) in <top 
(required)>'
      # ./spec/return_codes_spec.rb:10:in `chdir'
      # ./spec/return_codes_spec.rb:10:in `block (3 levels) in <top 
(required)>'

   2) return codes inside fixtures/frameworks when running 
testunit_bad.rb behaves like bad tests when print_error_status is 
disabled does not print anything to STDERR
      Failure/Error: expect(@stderr).to be_empty
        expected 
`"<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in 
`require': cannot 
loa...r_ruby/rubygems/core_ext/kernel_require.rb>:85:in 
`require'\n\tfrom testunit_bad.rb:2:in `<main>'\n".empty?` to be truthy, 
got false
      Shared Example Group: "bad tests" called from 
./spec/return_codes_spec.rb:66
      # ./spec/return_codes_spec.rb:49:in `block (5 levels) in <top 
(required)>'
      # ./spec/return_codes_spec.rb:12:in `block (4 levels) in <top 
(required)>'
      # ./spec/return_codes_spec.rb:10:in `chdir'
      # ./spec/return_codes_spec.rb:10:in `block (3 levels) in <top 
(required)>'

   3) return codes inside fixtures/frameworks when running rspec_good.rb 
behaves like good tests has a zero exit status
      Failure/Error: expect(@status.exitstatus).to be_zero
        expected `1.zero?` to be truthy, got false
      Shared Example Group: "good tests" called from 
./spec/return_codes_spec.rb:61
      # ./spec/return_codes_spec.rb:22:in `block (4 levels) in <top 
(required)>'
      # ./spec/return_codes_spec.rb:12:in `block (4 levels) in <top 
(required)>'
      # ./spec/return_codes_spec.rb:10:in `chdir'
      # ./spec/return_codes_spec.rb:10:in `block (3 levels) in <top 
(required)>'

   4) return codes inside fixtures/frameworks when running 
testunit_good.rb behaves like good tests has a zero exit status
      Failure/Error: expect(@status.exitstatus).to be_zero
        expected `1.zero?` to be truthy, got false
      Shared Example Group: "good tests" called from 
./spec/return_codes_spec.rb:56
      # ./spec/return_codes_spec.rb:22:in `block (4 levels) in <top 
(required)>'
      # ./spec/return_codes_spec.rb:12:in `block (4 levels) in <top 
(required)>'
      # ./spec/return_codes_spec.rb:10:in `chdir'
      # ./spec/return_codes_spec.rb:10:in `block (3 levels) in <top 
(required)>'

   5) return codes inside fixtures/frameworks when running 
testunit_good.rb behaves like good tests prints nothing to STDERR
      Failure/Error: expect(@stderr).to be_empty
        expected 
`"<internal:/usr/lib/ruby/vendor_ruby/rubygems/core_ext/kernel_require.rb>:85:in 
`require': cannot loa..._ruby/rubygems/core_ext/kernel_require.rb>:85:in 
`require'\n\tfrom testunit_good.rb:2:in `<main>'\n".empty?` to be 
truthy, got false
      Shared Example Group: "good tests" called from 
./spec/return_codes_spec.rb:56
      # ./spec/return_codes_spec.rb:26:in `block (4 levels) in <top 
(required)>'
      # ./spec/return_codes_spec.rb:12:in `block (4 levels) in <top 
(required)>'
      # ./spec/return_codes_spec.rb:10:in `chdir'
      # ./spec/return_codes_spec.rb:10:in `block (3 levels) in <top 
(required)>'

   6) return codes inside fixtures/frameworks when running rspec_bad.rb 
behaves like bad tests with default configuration prints a message to STDERR
      Failure/Error: expect(@stderr).to 
match(/stopped.+SimpleCov.+previous.+error/i)

        expected "" to match /stopped.+SimpleCov.+previous.+error/i
        Diff:
        @@ -1 +1 @@
        -/stopped.+SimpleCov.+previous.+error/i
        +""
      Shared Example Group: "bad tests" called from 
./spec/return_codes_spec.rb:71
      # ./spec/return_codes_spec.rb:37:in `block (5 levels) in <top 
(required)>'
      # ./spec/return_codes_spec.rb:12:in `block (4 levels) in <top 
(required)>'
      # ./spec/return_codes_spec.rb:10:in `chdir'
      # ./spec/return_codes_spec.rb:10:in `block (3 levels) in <top 
(required)>'

   7) A source file which is subsequently deleted does not cause an error
      Failure/Error: expect($?.exitstatus).to be_zero
        expected `1.zero?` to be truthy, got false
      # ./spec/deleted_source_spec.rb:11:in `block (3 levels) in <top 
(required)>'
      # ./spec/deleted_source_spec.rb:9:in `chdir'
      # ./spec/deleted_source_spec.rb:9:in `block (2 levels) in <top 
(required)>'

Finished in 3.63 seconds (files took 0.27945 seconds to load)
385 examples, 7 failures

Failed examples:

rspec ./spec/return_codes_spec.rb[1:1:3:1:1:2] # return codes inside 
fixtures/frameworks when running testunit_bad.rb behaves like bad tests 
with default configuration prints a message to STDERR
rspec ./spec/return_codes_spec.rb[1:1:3:1:2:2] # return codes inside 
fixtures/frameworks when running testunit_bad.rb behaves like bad tests 
when print_error_status is disabled does not print anything to STDERR
rspec ./spec/return_codes_spec.rb[1:1:2:1:1] # return codes inside 
fixtures/frameworks when running rspec_good.rb behaves like good tests 
has a zero exit status
rspec ./spec/return_codes_spec.rb[1:1:1:1:1] # return codes inside 
fixtures/frameworks when running testunit_good.rb behaves like good 
tests has a zero exit status
rspec ./spec/return_codes_spec.rb[1:1:1:1:2] # return codes inside 
fixtures/frameworks when running testunit_good.rb behaves like good 
tests prints nothing to STDERR
rspec ./spec/return_codes_spec.rb[1:1:4:1:1:2] # return codes inside 
fixtures/frameworks when running rspec_bad.rb behaves like bad tests 
with default configuration prints a message to STDERR
rspec ./spec/deleted_source_spec.rb:8 # A source file which is 
subsequently deleted does not cause an error

Randomized with seed 41104

/usr/bin/ruby3.0 
-I/usr/share/rubygems-integration/all/gems/rspec-support-3.10.3/lib:/usr/share/rubygems-integration/all/gems/rspec-core-3.10.1/lib 
/usr/share/rubygems-integration/all/gems/rspec-core-3.10.1/exe/rspec 
--pattern ./spec/\*\*/\*_spec.rb  --exclude-pattern 
./spec/default_formatter_spec.rb,./spec/gemspec_spec.rb --format 
documentation failed
mv ./.gem2deb.lib lib
mv ./.gem2deb.Gemfile.lock Gemfile.lock
mv test_projects/monorepo/.gem2deb.Gemfile.lock 
test_projects/monorepo/Gemfile.lock
mv test_projects/parallel_tests/.gem2deb.Gemfile.lock 
test_projects/parallel_tests/Gemfile.lock
mv test_projects/rails/rspec_rails/.gem2deb.Gemfile.lock 
test_projects/rails/rspec_rails/Gemfile.lock
autopkgtest [07:20:32]: test gem2deb-test-runner

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 495 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/pkg-ruby-extras-maintainers/attachments/20220324/54a6c0fc/attachment.sig>

More information about the Pkg-ruby-extras-maintainers mailing list