[Debian-science-sagemath] Doctests leaking file handles

[Tobias Hansen]

Hi,

the problem is still there with sage 9.1 (which I pushed to the Debian packaging repo). Since the number of tested files before the error is similar to the file descriptor limit, it seems plausible that each DocTestWorker in sage/doctest/forker.py is leaking one pipe. I saw that there are many open pipes by looking at /proc/$PID/fd/, however this does not give me any additional info about these pipes.

Unfortunately without deeper knowledge about forking in Python, I don't see what causes the leak. Could anyone please have a look at forker.py to help me?

Best,
Tobias

On 5/3/20 6:14 PM, Tobias Hansen wrote:
> Hi,
>
> I am having problems with a file handle leak during doctesting in the sagemath Debian package. This already occurred sporadically with sage 9.0 but with 9.1 (rc2) it happens more consistently. When the file handle limit (ulimit -n) is reached, the remaining tests are killed and one gets this:
>
> Traceback (most recent call last):
>   File "/home/thansen/src/sage/sagemath/sagemath/sage/src/bin/sage-runtests", line 178, in <module>
>     err = DC.run()
>   File "/home/thansen/src/sage/sagemath/sagemath/debian/build/usr/lib/python3/dist-packages/sage/doctest/control.py", line 1227, in run
>     self.run_doctests()
>   File "/home/thansen/src/sage/sagemath/sagemath/debian/build/usr/lib/python3/dist-packages/sage/doctest/control.py", line 928, in run_doctests
>     self.dispatcher.dispatch()
>   File "/home/thansen/src/sage/sagemath/sagemath/debian/build/usr/lib/python3/dist-packages/sage/doctest/forker.py", line 2024, in dispatch
>     self.parallel_dispatch()
>   File "/home/thansen/src/sage/sagemath/sagemath/debian/build/usr/lib/python3/dist-packages/sage/doctest/forker.py", line 1916, in parallel_dispatch
>     w.start()  # This might take some time
>   File "/home/thansen/src/sage/sagemath/sagemath/debian/build/usr/lib/python3/dist-packages/sage/doctest/forker.py", line 2191, in start
>     super(DocTestWorker, self).start()
>   File "/usr/lib/python3.8/multiprocessing/process.py", line 121, in start
>     self._popen = self._Popen(self)
>   File "/usr/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
>     return _default_context.get_context().Process._Popen(process_obj)
>   File "/usr/lib/python3.8/multiprocessing/context.py", line 276, in _Popen
>     return Popen(process_obj)
>   File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
>     self._launch(process_obj)
>   File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 69, in _launch
>     child_r, parent_w = os.pipe()
> OSError: [Errno 24] Too many open files
>
> The number of tests that run before this is similar to the file handle limit (e.g. I saw 1234 tests for a limit of 1024 and 103 tests for a limit of 100).
>
> I don't know if this comes from our patches, an external program or sage itself. At least our patches change only doctest results in sage/doctest and I didn't find anything suspicious elsewhere. It could also be that it happens only with python 3.8 which is not in sage yet. Has anyone else seen this? Any idea where to look or how to debug this?
>
> Best,
>
> Tobias
>
>