[Aptitude-devel] Bug#826783: libapt-pkg5.0: aptitude aborts due to a fd leak

[David Kalnischkies]

Control: reassign -1 aptitude

On Thu, Jun 09, 2016 at 03:42:30AM +0200, Jiri Palecek wrote:
> Then I looked at the file descriptors that were open:
[…]
> aptitude 2856 root    7r   REG    8,6     7692 2065017 /var/lib/apt/lists/ftp.cz.debian.org_debian_dists_experimental_non-free_i18n_Translation-en.lz4
> aptitude 2856 root    8r   REG    8,6     9603 2065060 /var/lib/apt/lists/ftp.cz.debian.org_debian_dists_experimental_non-free_binary-i386_Packages.lz4
> aptitude 2856 root    9r   REG    8,6     4606 2065265 /var/lib/apt/lists/ftp.cz.debian.org_debian_dists_experimental_contrib_i18n_Translation-en.lz4
> aptitude 2856 root   10r   REG    8,6     6224 2065213 /var/lib/apt/lists/ftp.cz.debian.org_debian_dists_experimental_contrib_binary-i386_Packages.lz4
[… more lz4 files …]
> It seems something in aptitude or libapt-pkg is opening those files, but
> not closing them. These file descriptors are then missing for
> downloading new packages. Could you look into it?

So, I went hunting yesterday for FD leaks in libapt and found a few…

Three of them quite boring from a user point of view, but one of them is
a leak in the lz4 (de)compression code which explains many of the leaked
fds in your output as you have compressed indexes enabled & lz4 is the
currently lowest cost compression type – but that output also contains
a few uncompressed files (like the dpkg/status file) multiple times
which I can't explain & haven't seen with any apt tool so far and
I ended up running our testcases with strategically placed:

	auto const pid = getpid();
	std::string ls;
	strprintf(ls, "ls -l /proc/%d/fd >&2", pid);
	system(ls.c_str());

so those leaks (if they are the fault of libapt) are caused by how
aptitude is calling it which apt tools don't do, so I reassign this bug
for now over to them so they can investigate were these leaks happen in
the hope that we can then figure out if its misuse of libapt or "just"
uncommon use causing a leak as that is likely more efficient than
letting me wade through aptitude code on my own…

(as the lz4 leak is by far the biggest eater of fds in your case this
bug could be closed with it, but we gonna catch them all…)


Best regards

David Kalnischkies
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/aptitude-devel/attachments/20160610/4e98a574/attachment.sig>