[Python-modules-team] Bug#921803: python-scrapy: FTBFS (failing tests)

[Santiago Vila]

Package: src:python-scrapy
Version: 1.5.1-1
Severity: serious
Tags: ftbfs

Dear maintainer:

I tried to build this package in buster but it failed:

--------------------------------------------------------------------------------
[...]
 debian/rules build-indep
dh build-indep --with python2,python3,bash_completion,sphinxdoc --buildsystem=pybuild
   dh_update_autotools_config -i -O--buildsystem=pybuild
   dh_autoreconf -i -O--buildsystem=pybuild
   dh_auto_configure -i -O--buildsystem=pybuild
I: pybuild base:217: python2.7 setup.py config 
running config
I: pybuild base:217: python3.7 setup.py config 
running config
   dh_auto_build -i -O--buildsystem=pybuild
I: pybuild base:217: /usr/bin/python setup.py build 
running build
running build_py
creating /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build/scrapy

[... snipped ...]

  /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build/tests/test_linkextractors.py:24: ScrapyDeprecationWarning: SgmlLinkExtractor is deprecated and will be removed in future releases. Please use scrapy.linkextractors.LinkExtractor
    lx = self.extractor_cls()

.pybuild/cpython2_2.7_scrapy/build/tests/test_linkextractors_deprecated.py::SgmlLinkExtractorTestCase::test_xhtml
  /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build/tests/test_linkextractors.py:421: ScrapyDeprecationWarning: SgmlLinkExtractor is deprecated and will be removed in future releases. Please use scrapy.linkextractors.LinkExtractor
    lx = self.extractor_cls()
  /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build/tests/test_linkextractors.py:432: ScrapyDeprecationWarning: SgmlLinkExtractor is deprecated and will be removed in future releases. Please use scrapy.linkextractors.LinkExtractor
    lx = self.extractor_cls()

.pybuild/cpython2_2.7_scrapy/build/tests/test_linkextractors_deprecated.py::RegexLinkExtractorTestCase::test_html_base_href
  /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build/tests/test_linkextractors_deprecated.py:228: ScrapyDeprecationWarning: SgmlLinkExtractor is deprecated and will be removed in future releases. Please use scrapy.linkextractors.LinkExtractor
    lx = RegexLinkExtractor()

.pybuild/cpython2_2.7_scrapy/build/tests/test_linkextractors_deprecated.py::RegexLinkExtractorTestCase::test_link_wrong_href
  /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build/tests/test_linkextractors_deprecated.py:210: ScrapyDeprecationWarning: SgmlLinkExtractor is deprecated and will be removed in future releases. Please use scrapy.linkextractors.LinkExtractor
    lx = RegexLinkExtractor()

.pybuild/cpython2_2.7_scrapy/build/tests/test_spider.py::CrawlSpiderTest::test_follow_links_attribute_deprecated_population
  /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build/scrapy/spiders/crawl.py:106: ScrapyDeprecationWarning: set_crawler is deprecated, instantiate and bound the spider to this crawler with from_crawler method instead.
    super(CrawlSpider, self).set_crawler(crawler)
  /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build/scrapy/spiders/crawl.py:106: ScrapyDeprecationWarning: set_crawler is deprecated, instantiate and bound the spider to this crawler with from_crawler method instead.
    super(CrawlSpider, self).set_crawler(crawler)

.pybuild/cpython2_2.7_scrapy/build/tests/test_utils_deprecate.py::WarnWhenSubclassedTest::test_warning_on_instance
  /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build/tests/test_utils_deprecate.py:115: MyWarning: tests.test_utils_deprecate.UserClass inherits from deprecated class tests.test_utils_deprecate.Deprecated, please inherit from tests.test_utils_deprecate.NewName. (warning only on first subclass, there may be others)
    class UserClass(Deprecated):

.pybuild/cpython2_2.7_scrapy/build/tests/test_utils_python.py::UtilsPythonTestCase::test_stringify_dict
  /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build/tests/test_utils_python.py:157: ScrapyDeprecationWarning: Call to deprecated function stringify_dict.
    d2 = stringify_dict(d, keys_only=False)

.pybuild/cpython2_2.7_scrapy/build/tests/test_utils_python.py::UtilsPythonTestCase::test_stringify_dict_keys_only
  /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build/tests/test_utils_python.py:176: ScrapyDeprecationWarning: Call to deprecated function stringify_dict.
    d2 = stringify_dict(d)

.pybuild/cpython2_2.7_scrapy/build/tests/test_utils_python.py::UtilsPythonTestCase::test_stringify_dict_tuples
  /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build/tests/test_utils_python.py:167: ScrapyDeprecationWarning: Call to deprecated function stringify_dict.
    d2 = stringify_dict(tuples, keys_only=False)

-- Docs: https://docs.pytest.org/en/latest/warnings.html
 1 failed, 1648 passed, 13 skipped, 14 xfailed, 427 warnings in 373.86 seconds =
E: pybuild pybuild:338: test: plugin distutils failed with: exit code=1: cd /<<PKGBUILDDIR>>/.pybuild/cpython2_2.7_scrapy/build; python2.7 -m pytest tests
dh_auto_test: pybuild --test --test-pytest -i python{version} -p 2.7 returned exit code 13
make: *** [debian/rules:9: build-indep] Error 25
dpkg-buildpackage: error: debian/rules build-indep subprocess returned exit status 2
--------------------------------------------------------------------------------

(The above is just how the build ends and not necessarily the most relevant part)

The build was made in my autobuilder with "dpkg-buildpackage -A"
and it also fails here:

https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/python-scrapy.html

where you can get a full build log if you need it.

If this is really a bug in one of the build-depends, please use reassign and affects,
so that this is still visible in the BTS web page for this package.

Thanks.