Bug#1023834: shared-mime-info: update-mime-database takes 30-40 minutes to run

Thu Nov 10 23:45:17 GMT 2022

Package: shared-mime-info
Version: 2.0-1
Severity: important
Tags: upstream
X-Debbugs-Cc: RossBoylan at stanfordalumni.org

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Steps to Reproduce
==================
1. btrfs on top of encrypted partition on a spinning disk. btrfs subvolumes in
use.
2. sources.list.d includes entry for Visual Studio code (deb [arch=amd64
signed-by=/usr/share/keyrings/microsoft-archive-keyring.gpg]
https://packages.microsoft.com/repos/vscode stable main).
3. aptitude shows an update to code is available.
4. apply update and wait, as much as 45 minutes.  Most of the time update-mime-
database is consuming 50-90% of one CPU.

This happens every time, even for the most minor update.

I often am recording TV shows to the disk at the same time, but this morning I
was not and it took 30 minutes.  Other times it has been 40-45 minutes.

Comments
========

I have opened a bug against vscode upstream
(https://github.com/microsoft/vscode/issues/163690), and it seems likely their
settings are not optimal.  In particular, their postinst script invokes update-
mime-database without the -n option.  I do not usually experience such long
delays when installing/upgrading other packages.

I believe -n causes update-mime-database to skip scanning entirely if nothing
seems new--the man page is a bit unclear.  So perhaps that one fix would take
care of most of my problems.

However, when update-mime-database does a real run, it seems to me it should be
much faster.  This appears to be a long-standing problem:

https://bugs.freedesktop.org/show_bug.cgi?id=70366 (also
https://bugs.gentoo.org/487504, https://bugs.archlinux.org/task/66687,
https://bugs.launchpad.net/ubuntu/+source/shared-mime-info/+bug/1816674).

Some of the bugs have been closed, but I think in all cases that was because a
facility to disable fsync() via an environment variable was added and the build
or package installation system was modified to use it.  In contrast, the
performance I'm seeing strongly suggests it remains enabled on Debian.

I spent some time trying to figure out how set the PKGSYSTEM_ENABLE_FSYNC
environment variable so that it would be applied whenever update-mime-database
ran.  The answer is not obvious to me.  The man page mentions no configuration
files, and a quick look at main() in the source doesn't seem to show any being
read in.  main() does check the environment variable.  I also looked for ways
to set the variable in dpkg or apt, but didn't find anything (probably there is
a way).

Desired Solutions
=================

In increasing order of difficulty/impact:

1. Document the PKGSYSTEM_ENABLE_FSYNC environment variable in the man page.
2. Describe how the user can disable the fsync during upgrades, bearing in mind
that the invocation of update-mime-database is indirect.
3. Turn fsync off for update-mime-database as a default.

I completely agree with Chantziaras's argument
(https://bugs.freedesktop.org/show_bug.cgi?id=70366#c10) that using fsync on a
database that can easily be rebuilt if things go wrong is unjustified.  And, as
noted above, disabling the fsync has been the common response across the
various forms of packaging and build systems.

Coda
====

The extremely poor performance I'm seeing is likely the result of failings in
both the code packaging--not an official Debian package--and some weaknesses in
btrfs.  But problems with fsync in update-mime-database are widely reported,
and it would be good if Debian could do something to mitigate them.

- -- System Information:
Debian Release: 11.5
  APT prefers stable-security
  APT policy: (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.10.0-19-amd64 (SMP w/8 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE,
TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not
set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages shared-mime-info depends on:
ii  libc6         2.31-13+deb11u4
ii  libglib2.0-0  2.66.8-1
ii  libxml2       2.9.10+dfsg-6.7+deb11u3

shared-mime-info recommends no packages.

shared-mime-info suggests no packages.