[Python-apps-team] Bug#855092: beets: FTBFS randomly (failing tests)

[Carl Suster]

Hi,

I'm an upstream contributor to beets and I was looking into the failures 
you're seeing here. I'm interested in making these tests reliable.

I tried to build the package on my (sid) laptop using sbuild and the 
latest packaging repo from salsa. I'm not able to reproduce these test 
failures. If I remove the Debian patches disabling tests, I'm still not 
able to reproduce any of the failures that led you to add those patches 
in the first case. I'm seeing three categories of tests here:

   1) The test in skip-broken-test. If the failure you're seeing is the 
same error on the GitHub issue you mention in that patch 
('musicbrainz.host'), then my suspicion is that when running the test 
beets is unable to find/read the file beets/config_default.yaml. One way 
this can happen is if beets is being invoked as a zipped egg rather than 
unpacked source (unsupported). Otherwise it might be that the build 
environment has paths set in an unusual way that interferes with beets' 
mechanism to find that YAML file relative to the invoked module.

   2) There are two tests failing due to filesystem access 
(test_no_write_permission and test_add_tags). Maybe we can do a better 
job of mocking here so that the actual filesystem isn't being tested. 
I'll take a look.

   3) The two test_import_task_created* tests exercise a feature based 
on a coroutine implementation (beets.util.pipeline), so I wonder if 
that's related? It's the only unusual thing I can think of off-hand. I 
know that the Debian build infrastructure is a little unusual, but I'm 
not sure what specifically the difference could be here.

If you can help point me in the right direction to reproduce these 
issues that would be appreciated.

Cheers,
Carl