[Debian-med-packaging] Bug#1031192: toil: FTBFS (The job JobClass is requesting 2.0 cores)
Santiago Vila
sanvila at debian.org
Mon Feb 13 00:29:33 GMT 2023
Package: src:toil
Version: 5.9.2-2
Severity: important
Tags: ftbfs
Dear maintainer:
During a rebuild of all packages in bookworm, your package failed to build:
--------------------------------------------------------------------------------
[...]
debian/rules binary-indep
dh binary-indep --with python3 --buildsystem=pybuild
dh_update_autotools_config -i -O--buildsystem=pybuild
dh_autoreconf -i -O--buildsystem=pybuild
dh_auto_configure -i -O--buildsystem=pybuild
I: pybuild base:240: python3.11 setup.py config
/<<PKGBUILDDIR>>/setup.py:14: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses
import imp
running config
debian/rules override_dh_auto_build
make[1]: Entering directory '/<<PKGBUILDDIR>>'
dh_auto_build
I: pybuild base:240: /usr/bin/python3 setup.py build
/<<PKGBUILDDIR>>/setup.py:14: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses
[... snipped ...]
[gw0] [ 86%] PASSED src/toil/test/src/regularLogTest.py::RegularLogTest::testWriteLogs
src/toil/test/src/resourceTest.py::ResourceTest::testBuiltIn
[gw0] [ 87%] PASSED src/toil/test/src/resourceTest.py::ResourceTest::testBuiltIn
src/toil/test/src/resourceTest.py::ResourceTest::testNonPyStandAlone
[gw0] [ 87%] PASSED src/toil/test/src/resourceTest.py::ResourceTest::testNonPyStandAlone
src/toil/test/src/resourceTest.py::ResourceTest::testPackage
[gw0] [ 87%] PASSED src/toil/test/src/resourceTest.py::ResourceTest::testPackage
src/toil/test/src/resourceTest.py::ResourceTest::testStandAlone
[gw0] [ 87%] PASSED src/toil/test/src/resourceTest.py::ResourceTest::testStandAlone
src/toil/test/src/resourceTest.py::ResourceTest::testStandAloneInPackage
[gw0] [ 87%] PASSED src/toil/test/src/resourceTest.py::ResourceTest::testStandAloneInPackage
src/toil/test/src/restartDAGTest.py::RestartDAGTest::testRestartedWorkflowSchedulesCorrectJobsOnFailedParent
[gw0] [ 87%] SKIPPED src/toil/test/src/restartDAGTest.py::RestartDAGTest::testRestartedWorkflowSchedulesCorrectJobsOnFailedParent
src/toil/test/src/restartDAGTest.py::RestartDAGTest::testRestartedWorkflowSchedulesCorrectJobsOnKilledParent
[gw0] [ 88%] SKIPPED src/toil/test/src/restartDAGTest.py::RestartDAGTest::testRestartedWorkflowSchedulesCorrectJobsOnKilledParent
src/toil/test/src/resumabilityTest.py::ResumabilityTest::test
[gw0] [ 88%] SKIPPED src/toil/test/src/resumabilityTest.py::ResumabilityTest::test
src/toil/test/src/systemTest.py::SystemTest::testAtomicityOfNonEmptyDirectoryRenames
[gw0] [ 88%] PASSED src/toil/test/src/systemTest.py::SystemTest::testAtomicityOfNonEmptyDirectoryRenames
src/toil/test/src/threadingTest.py::ThreadingTest::testGlobalMutexOrdering
[gw0] [ 88%] PASSED src/toil/test/src/threadingTest.py::ThreadingTest::testGlobalMutexOrdering
src/toil/test/src/threadingTest.py::ThreadingTest::testLastProcessStanding
[gw0] [ 88%] PASSED src/toil/test/src/threadingTest.py::ThreadingTest::testLastProcessStanding
src/toil/test/src/toilContextManagerTest.py::ToilContextManagerTest::testContextManger
[gw0] [ 89%] SKIPPED src/toil/test/src/toilContextManagerTest.py::ToilContextManagerTest::testContextManger
src/toil/test/src/toilContextManagerTest.py::ToilContextManagerTest::testExportAfterFailedExport
[gw0] [ 89%] SKIPPED src/toil/test/src/toilContextManagerTest.py::ToilContextManagerTest::testExportAfterFailedExport
src/toil/test/src/toilContextManagerTest.py::ToilContextManagerTest::testNoContextManger
[gw0] [ 89%] SKIPPED src/toil/test/src/toilContextManagerTest.py::ToilContextManagerTest::testNoContextManger
src/toil/test/src/userDefinedJobArgTypeTest.py::UserDefinedJobArgTypeTest::testJobClass
[gw0] [ 89%] SKIPPED src/toil/test/src/userDefinedJobArgTypeTest.py::UserDefinedJobArgTypeTest::testJobClass
src/toil/test/src/userDefinedJobArgTypeTest.py::UserDefinedJobArgTypeTest::testJobClassFromMain
[gw0] [ 89%] PASSED src/toil/test/src/userDefinedJobArgTypeTest.py::UserDefinedJobArgTypeTest::testJobClassFromMain
src/toil/test/src/userDefinedJobArgTypeTest.py::UserDefinedJobArgTypeTest::testJobFunction
[gw0] [ 89%] FAILED src/toil/test/src/userDefinedJobArgTypeTest.py::UserDefinedJobArgTypeTest::testJobFunction
src/toil/test/src/userDefinedJobArgTypeTest.py::UserDefinedJobArgTypeTest::testJobFunctionFromMain
[gw0] [ 90%] PASSED src/toil/test/src/userDefinedJobArgTypeTest.py::UserDefinedJobArgTypeTest::testJobFunctionFromMain
src/toil/test/src/workerTest.py::WorkerTests::testNextChainable
[gw0] [ 90%] PASSED src/toil/test/src/workerTest.py::WorkerTests::testNextChainable
src/toil/test/utils/toilDebugTest.py::testJobStoreContents
[gw0] [ 90%] SKIPPED src/toil/test/utils/toilDebugTest.py::testJobStoreContents
src/toil/test/utils/toilDebugTest.py::testFetchJobStoreFiles
[gw0] [ 90%] PASSED src/toil/test/utils/toilDebugTest.py::testFetchJobStoreFiles
src/toil/test/utils/toilDebugTest.py::testFetchJobStoreFilesWSymlinks
[gw0] [ 90%] PASSED src/toil/test/utils/toilDebugTest.py::testFetchJobStoreFilesWSymlinks
src/toil/test/utils/utilsTest.py::UtilsTest::testAWSProvisionerUtils
[gw0] [ 91%] SKIPPED src/toil/test/utils/utilsTest.py::UtilsTest::testAWSProvisionerUtils
src/toil/test/utils/utilsTest.py::UtilsTest::testGetPIDStatus
[gw0] [ 91%] PASSED src/toil/test/utils/utilsTest.py::UtilsTest::testGetPIDStatus
src/toil/test/utils/utilsTest.py::UtilsTest::testGetStatusFailedCWLWF
[gw0] [ 91%] SKIPPED src/toil/test/utils/utilsTest.py::UtilsTest::testGetStatusFailedCWLWF
src/toil/test/utils/utilsTest.py::UtilsTest::testGetStatusFailedToilWF
[gw0] [ 91%] PASSED src/toil/test/utils/utilsTest.py::UtilsTest::testGetStatusFailedToilWF
src/toil/test/utils/utilsTest.py::UtilsTest::testGetStatusSuccessfulCWLWF
[gw0] [ 91%] SKIPPED src/toil/test/utils/utilsTest.py::UtilsTest::testGetStatusSuccessfulCWLWF
src/toil/test/utils/utilsTest.py::UtilsTest::testMultipleJobsPerWorkerStats
[gw0] [ 91%] SKIPPED src/toil/test/utils/utilsTest.py::UtilsTest::testMultipleJobsPerWorkerStats
src/toil/test/utils/utilsTest.py::UtilsTest::testPrintJobLog
[gw0] [ 92%] PASSED src/toil/test/utils/utilsTest.py::UtilsTest::testPrintJobLog
src/toil/test/utils/utilsTest.py::UtilsTest::testRestartAttribute
[gw0] [ 92%] PASSED src/toil/test/utils/utilsTest.py::UtilsTest::testRestartAttribute
src/toil/test/utils/utilsTest.py::UtilsTest::testUnicodeSupport
[gw0] [ 92%] PASSED src/toil/test/utils/utilsTest.py::UtilsTest::testUnicodeSupport
src/toil/test/utils/utilsTest.py::UtilsTest::testUtilsSort
[gw0] [ 92%] SKIPPED src/toil/test/utils/utilsTest.py::UtilsTest::testUtilsSort
src/toil/test/utils/utilsTest.py::UtilsTest::testUtilsStatsSort
[gw0] [ 92%] SKIPPED src/toil/test/utils/utilsTest.py::UtilsTest::testUtilsStatsSort
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Ceil
[gw0] [ 93%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Ceil
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Cross
[gw0] [ 93%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Cross
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Floor
[gw0] [ 93%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Floor
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Length
[gw0] [ 93%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Length
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadBoolean
[gw0] [ 93%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadBoolean
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadFloat
[gw0] [ 93%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadFloat
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadInt
[gw0] [ 94%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadInt
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadJson
[gw0] [ 94%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadJson
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadLines
[gw0] [ 94%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadLines
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadMap
[gw0] [ 94%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadMap
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadString
[gw0] [ 94%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadString
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadTsv
[gw0] [ 95%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_ReadTsv
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Sub
[gw0] [ 95%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Sub
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Transpose
[gw0] [ 95%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Transpose
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_WriteJson
[gw0] [ 95%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_WriteJson
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_WriteLines
[gw0] [ 95%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_WriteLines
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_WriteMap
[gw0] [ 95%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_WriteMap
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_WriteTsv
[gw0] [ 96%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_WriteTsv
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Zip
[gw0] [ 96%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryFunctionsTest::testFn_Zip
src/toil/test/wdl/builtinTest.py::WdlLanguageSpecWorkflowsTest::test_type_pair
[gw0] [ 96%] PASSED src/toil/test/wdl/builtinTest.py::WdlLanguageSpecWorkflowsTest::test_type_pair
src/toil/test/wdl/builtinTest.py::WdlLanguageSpecWorkflowsTest::test_v1_declaration
[gw0] [ 96%] PASSED src/toil/test/wdl/builtinTest.py::WdlLanguageSpecWorkflowsTest::test_v1_declaration
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_as_map
[gw0] [ 96%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_as_map
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_as_pairs
[gw0] [ 97%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_as_pairs
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_ceil
[gw0] [ 97%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_ceil
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_collect_by_key
[gw0] [ 97%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_collect_by_key
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_cross
[gw0] [ 97%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_cross
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_flatten
[gw0] [ 97%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_flatten
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_floor
[gw0] [ 97%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_floor
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_keys
[gw0] [ 98%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_keys
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_length
[gw0] [ 98%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_length
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_range
[gw0] [ 98%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_range
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_read
[gw0] [ 98%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_read
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_round
[gw0] [ 98%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_round
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_size
[gw0] [ 99%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_size
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_stdout
[gw0] [ 99%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_stdout
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_sub
[gw0] [ 99%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_sub
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_transpose
[gw0] [ 99%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_transpose
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_write
[gw0] [ 99%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_write
src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_zip
[gw0] [100%] PASSED src/toil/test/wdl/builtinTest.py::WdlStandardLibraryWorkflowsTest::test_zip
=================================== FAILURES ===================================
________________________ HelloWorldTest.testHelloWorld _________________________
[gw0] linux -- Python 3.11.1 /usr/bin/python3.11
Traceback (most recent call last):
File "/<<PKGBUILDDIR>>/src/toil/test/src/helloWorldTest.py", line 23, in testHelloWorld
Job.Runner.startToil(HelloWorld(), options)
File "/<<PKGBUILDDIR>>/src/toil/job.py", line 2131, in startToil
return toil.start(job)
^^^^^^^^^^^^^^^
File "/<<PKGBUILDDIR>>/src/toil/common.py", line 1030, in start
return self._runMainLoop(rootJobDescription)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/<<PKGBUILDDIR>>/src/toil/common.py", line 1475, in _runMainLoop
jobCache=self._jobCache).run()
^^^^^
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 292, in run
self.innerLoop()
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 778, in innerLoop
self._processReadyJobs()
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 675, in _processReadyJobs
self._processReadyJob(message.job_id, message.result_status)
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 571, in _processReadyJob
self.issueJob(readyJob)
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 939, in issueJob
jobBatchSystemID = self.batchSystem.issueBatchJob(jobNode, job_environment=job_environment)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/singleMachine.py", line 731, in issueBatchJob
self.check_resource_request(scaled_desc)
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/singleMachine.py", line 490, in check_resource_request
raise e
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/singleMachine.py", line 486, in check_resource_request
super().check_resource_request(requirer)
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/abstractBatchSystem.py", line 335, in check_resource_request
raise e
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/abstractBatchSystem.py", line 328, in check_resource_request
raise InsufficientSystemResources(requirer, resource, available)
toil.batchSystems.abstractBatchSystem.InsufficientSystemResources: The job HelloWorld is requesting 2.0 cores, more than the maximum of 1 cores that SingleMachineBatchSystem was configured with, or enforced by --maxCores. Scale is set to 1.0.
---------------------------- Captured stdout setup -----------------------------
[TEST] toil.test.src.helloWorldTest.HelloWorldTest:testHelloWorld (Feb 09 2023 05:57:43:791834 PST)
------------------------------ Captured log call -------------------------------
INFO toil.test:__init__.py:114 Setting up toil.test.src.helloWorldTest.HelloWorldTest.testHelloWorld ...
DEBUG toil.batchSystems.options:options.py:166 Add options for <class 'toil.batchSystems.awsBatch.AWSBatchBatchSystem'>
DEBUG toil.batchSystems.options:options.py:166 Add options for <class 'toil.batchSystems.parasol.ParasolBatchSystem'>
DEBUG toil.batchSystems.options:options.py:166 Add options for <class 'toil.batchSystems.singleMachine.SingleMachineBatchSystem'>
DEBUG toil.batchSystems.options:options.py:166 Add options for <class 'toil.batchSystems.gridengine.GridEngineBatchSystem'>
DEBUG toil.batchSystems.options:options.py:166 Add options for <class 'toil.batchSystems.lsf.LSFBatchSystem'>
DEBUG toil.batchSystems.options:options.py:166 Add options for <class 'toil.batchSystems.slurm.SlurmBatchSystem'>
DEBUG toil.batchSystems.options:options.py:166 Add options for <class 'toil.batchSystems.torque.TorqueBatchSystem'>
DEBUG toil.resource:resource.py:416 Module dir is /<<PKGBUILDDIR>>/src, our prefix is /usr, virtualenv: False
INFO toil.job:job.py:2532 Saving graph of 1 jobs, 1 new
INFO toil.job:job.py:2544 Processing job 'HelloWorld' kind-HelloWorld/instance-s7485mrp v0
INFO toil:__init__.py:442 Running Toil version 5.9.0-8155e0a981f4d728762a7cbc920b7ed544fe4ae7 on host skywalker2.
INFO toil.test:__init__.py:119 Tore down toil.test.src.helloWorldTest.HelloWorldTest.testHelloWorld
____________________ RealtimeLoggerTest.testRealtimeLogger _____________________
[gw0] linux -- Python 3.11.1 /usr/bin/python3.11
Traceback (most recent call last):
File "/<<PKGBUILDDIR>>/src/toil/test/src/realtimeLoggerTest.py", line 33, in testRealtimeLogger
Job.Runner.startToil(LogTest(), options)
File "/<<PKGBUILDDIR>>/src/toil/job.py", line 2131, in startToil
return toil.start(job)
^^^^^^^^^^^^^^^
File "/<<PKGBUILDDIR>>/src/toil/common.py", line 1030, in start
return self._runMainLoop(rootJobDescription)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/<<PKGBUILDDIR>>/src/toil/common.py", line 1475, in _runMainLoop
jobCache=self._jobCache).run()
^^^^^
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 292, in run
self.innerLoop()
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 778, in innerLoop
self._processReadyJobs()
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 675, in _processReadyJobs
self._processReadyJob(message.job_id, message.result_status)
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 571, in _processReadyJob
self.issueJob(readyJob)
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 939, in issueJob
jobBatchSystemID = self.batchSystem.issueBatchJob(jobNode, job_environment=job_environment)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/singleMachine.py", line 731, in issueBatchJob
self.check_resource_request(scaled_desc)
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/singleMachine.py", line 490, in check_resource_request
raise e
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/singleMachine.py", line 486, in check_resource_request
super().check_resource_request(requirer)
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/abstractBatchSystem.py", line 335, in check_resource_request
raise e
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/abstractBatchSystem.py", line 328, in check_resource_request
raise InsufficientSystemResources(requirer, resource, available)
toil.batchSystems.abstractBatchSystem.InsufficientSystemResources: The job LogTest is requesting 2.0 cores, more than the maximum of 1 cores that SingleMachineBatchSystem was configured with, or enforced by --maxCores. Scale is set to 1.0. Scale is set to 1.0.
---------------------------- Captured stdout setup -----------------------------
[TEST] toil.test.src.realtimeLoggerTest.RealtimeLoggerTest:testRealtimeLogger (Feb 09 2023 05:58:31:604072 PST)
------------------------------ Captured log call -------------------------------
INFO toil.test:__init__.py:114 Setting up toil.test.src.realtimeLoggerTest.RealtimeLoggerTest.testRealtimeLogger ...
INFO toil.job:job.py:2532 Saving graph of 1 jobs, 1 new
INFO toil.job:job.py:2544 Processing job 'LogTest' kind-LogTest/instance-p2s_uza3 v0
INFO toil:__init__.py:442 Running Toil version 5.9.0-8155e0a981f4d728762a7cbc920b7ed544fe4ae7 on host skywalker2.
INFO toil.realtimeLogger:realtimeLogger.py:139 Starting real-time logging.
INFO toil.realtimeLogger:realtimeLogger.py:177 Stopping real-time logging server.
INFO toil.realtimeLogger:realtimeLogger.py:181 Joining real-time logging server thread.
INFO toil.test:__init__.py:119 Tore down toil.test.src.realtimeLoggerTest.RealtimeLoggerTest.testRealtimeLogger
__________________ UserDefinedJobArgTypeTest.testJobFunction ___________________
[gw0] linux -- Python 3.11.1 /usr/bin/python3.11
Traceback (most recent call last):
File "/<<PKGBUILDDIR>>/src/toil/test/src/userDefinedJobArgTypeTest.py", line 39, in testJobFunction
Job.Runner.startToil(Job.wrapJobFn(jobFunction, 0, Foo()), self.options)
File "/<<PKGBUILDDIR>>/src/toil/job.py", line 2131, in startToil
return toil.start(job)
^^^^^^^^^^^^^^^
File "/<<PKGBUILDDIR>>/src/toil/common.py", line 1030, in start
return self._runMainLoop(rootJobDescription)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/<<PKGBUILDDIR>>/src/toil/common.py", line 1475, in _runMainLoop
jobCache=self._jobCache).run()
^^^^^
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 292, in run
self.innerLoop()
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 778, in innerLoop
self._processReadyJobs()
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 675, in _processReadyJobs
self._processReadyJob(message.job_id, message.result_status)
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 591, in _processReadyJob
self._runJobSuccessors(job_id)
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 485, in _runJobSuccessors
self.issueJobs(successors)
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 962, in issueJobs
self.issueJob(job)
File "/<<PKGBUILDDIR>>/src/toil/leader.py", line 939, in issueJob
jobBatchSystemID = self.batchSystem.issueBatchJob(jobNode, job_environment=job_environment)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/singleMachine.py", line 731, in issueBatchJob
self.check_resource_request(scaled_desc)
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/singleMachine.py", line 490, in check_resource_request
raise e
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/singleMachine.py", line 486, in check_resource_request
super().check_resource_request(requirer)
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/abstractBatchSystem.py", line 335, in check_resource_request
raise e
File "/<<PKGBUILDDIR>>/src/toil/batchSystems/abstractBatchSystem.py", line 328, in check_resource_request
raise InsufficientSystemResources(requirer, resource, available)
toil.batchSystems.abstractBatchSystem.InsufficientSystemResources: The job JobClass is requesting 2.0 cores, more than the maximum of 1 cores that SingleMachineBatchSystem was configured with, or enforced by --maxCores. Scale is set to 1.0. Scale is set to 1.0. Scale is set to 1.0.
---------------------------- Captured stdout setup -----------------------------
[TEST] toil.test.src.userDefinedJobArgTypeTest.UserDefinedJobArgTypeTest:testJobFunction (Feb 09 2023 05:58:54:309071 PST)
----------------------------- Captured stderr call -----------------------------
You are using cwltool version 3.1.20230201224320, which might not be compatible with version 3.1.20221201130942 used by Toil. You should consider running 'pip install cwltool==3.1.20221201130942' to match Toil's cwltool version.
[2023-02-09T05:58:54+0000] [MainThread] [I] [toil.worker] Redirecting logging to /tmp/628d28235c38534299213a26d94e7d6f/66ee/worker_log.txt
------------------------------ Captured log call -------------------------------
INFO toil.test:__init__.py:114 Setting up toil.test.src.userDefinedJobArgTypeTest.UserDefinedJobArgTypeTest.testJobFunction ...
INFO toil.job:job.py:2532 Saving graph of 1 jobs, 1 new
INFO toil.job:job.py:2544 Processing job 'jobFunction' kind-jobFunction/instance-m3xy2axp v0
INFO toil:__init__.py:442 Running Toil version 5.9.0-8155e0a981f4d728762a7cbc920b7ed544fe4ae7 on host skywalker2.
INFO toil.leader:leader.py:949 Issued job 'jobFunction' kind-jobFunction/instance-m3xy2axp v1 with job batch system ID: 0 and disk: 2.0 Gi, memory: 2.0 Gi, cores: 1, accelerators: [], preemptible: False
INFO toil.leader:leader.py:1034 0 jobs are running, 0 jobs are issued and waiting to run
INFO toil.test:__init__.py:119 Tore down toil.test.src.userDefinedJobArgTypeTest.UserDefinedJobArgTypeTest.testJobFunction
=========================== short test summary info ============================
FAILED src/toil/test/src/helloWorldTest.py::HelloWorldTest::testHelloWorld - ...
FAILED src/toil/test/src/realtimeLoggerTest.py::RealtimeLoggerTest::testRealtimeLogger
FAILED src/toil/test/src/userDefinedJobArgTypeTest.py::UserDefinedJobArgTypeTest::testJobFunction
=========== 3 failed, 195 passed, 349 skipped in 1240.32s (0:20:40) ============
E: pybuild pybuild:388: test: plugin custom failed with: exit code=1: HOME=/<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_toil python3.11 setup.py develop --user && PYTHONPATH=/<<PKGBUILDDIR>>/src:$PYTHONPATH PATH=/<<PKGBUILDDIR>>/.pybuild/cpython3_3.11_toil/.local/bin/:$PATH TOIL_TEST_QUICK=True TOIL_SKIP_DOCKER=True python3.11 -m pytest -n auto --dist loadscope -vv -W ignore --ignore src/toil/test/lib/aws/test_s3.py --ignore src/toil/test/provisioners/aws/awsProvisionerTest.py --ignore src/toil/test/wdl/toilwdlTest.py --ignore src/toil/test/cwl/cwlTest.py --ignore src/toil/test/src/promisedRequirementTest.py --ignore src/toil/test/lib/test_ec2.py --ignore src/toil/test/batchSystems/batchSystemTest.py --ignore src/toil/test/lib/aws/test_iam.py -k "not (test_bioconda or test_run_conformance or testImportFtpFile or ToilWdlIntegrationTest or SortTest or testCwlexample or testVirtualEnv or ToilDocumentationTest or test_cwl_toil_kill or testImportReadFileCompatibility or CleanWorkDirTest or DeferredFunctionTest)" /<<PKGBUILDDIR>>/src/toil/test
dh_auto_test: error: pybuild --test --test-pytest -i python{version} -p 3.11 returned exit code 13
make[1]: *** [debian/rules:25: override_dh_auto_test] Error 25
make[1]: Leaving directory '/<<PKGBUILDDIR>>'
make: *** [debian/rules:11: binary-indep] Error 2
dpkg-buildpackage: error: debian/rules binary-indep subprocess returned exit status 2
--------------------------------------------------------------------------------
(The above is just how the build ends and not necessarily the most relevant part)
The build machine in this case had a single CPU, and I believe that's determinant
for the error to happen.
Therefore, to reproduce, please try first setting GRUB_CMDLINE_LINUX="nr_cpus=1"
in /etc/default/grub. If that does not work, then the general offer below
still applies.
I hope this is not too complex to fix. I tried setting cores=1 in
class JobClass at src/toil/test/src/userDefinedJobArgTypeTest.py,
but it was not enough.
---------------------------------------------------------------------
About the archive rebuild: The build was made using virtual machines
from Hetzner, with enough memory, enough disk, and either one or two
CPUs, using a reduced chroot with only build-essential packages (plus
debhelper).
If you could not reproduce the bug please contact me privately, as I
am willing to provide ssh access to a virtual machine where the bug is
fully reproducible.
If this is really a bug in one of the build-depends, please use
reassign and affects, so that this is still visible in the BTS web
page for this package.
Thanks.
More information about the Debian-med-packaging
mailing list