[Aptitude-devel] Bug#697724: aptitude: selecting packages in the curses interface slow iff Acquire::GzipIndexes="true"

[Axel Beckert]

Package: aptitude
Version: 0.6.8.2-1
Severity: normal

Hi,

this may be the cause for the commenter's issues at
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=317841#10 but is likely
not the cause for the issue of the original reporter of #317841 as there
were no .gz in the file names he reported.

If I set Acquire::GzipIndexes to "true" in apt.conf, remove all files
from /var/lib/apt/lists/ and run "apt-get update" again, the downloaded
files don't get uncompressed but stay in compressed form on disk.

When I now start aptitude's TUI and move the selection up and down with
the cursor keys (or click on a package with the mouse), it takes several
hundred milliseconds until I see a reaction from aptitude if the
selected line contains a package. It does not happen if I select a
package group or if I don't use compressed package files. I suspect this
is because aptitude needs to get the short description from the package
list, because it's displayed in the status bar upon package selection.

So it seems that aptitude uncompresses the package list which contains
the selected package each time I select a package. An "strace -efile" on
selecting the package initscripts with and without Acquire::GzipIndexes
shows the following amounts of stat64() calls on the package lists:

$ sort /tmp/aptitude-one-down-stat.strace | awk '{print $1}' | uniq -c | sort -n
[...]
      4 stat64("/var/lib/apt/lists/debian.ethz.ch_debian_dists_wheezy_non-free_binary-i386_Packages",
      4 stat64("/var/lib/apt/lists/debian.ethz.ch_debian_dists_wheezy_non-free_i18n_Translation-en",
     56 stat64("/var/lib/apt/lists/debian.ethz.ch_debian_dists_sid_main_binary-i386_Packages",
$ sort /tmp/aptitude-gzip-indexes-one-down-stat.strace | awk '{print $1}' | uniq -c | sort -n
[...]
      4 stat64("/var/lib/apt/lists/debian.ethz.ch_debian_dists_wheezy_non-free_binary-i386_Packages.gz",
      4 stat64("/var/lib/apt/lists/debian.ethz.ch_debian_dists_wheezy_non-free_i18n_Translation-en",
     27 stat64("/var/lib/apt/lists/debian.ethz.ch_debian_dists_sid_main_binary-i386_Packages",
     54 stat64("/var/lib/apt/lists/debian.ethz.ch_debian_dists_sid_main_binary-i386_Packages.gz",

So aptitude at least stat64()s several times on every package selection
if the uncompressed package lists.


I though don't expect that this causes the obvious speed difference
between with and without Acquire::GzipIndexes, so I suspect it's some
decompression which slows down selecting packages in the TUI.

So maybe it can cache the information gathered at the first time read or
read all the information into memory on startup (which would likely
raise memory consumption which would be not ideal either).

-- Package-specific info:
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: not available
URL: <http://lists.alioth.debian.org/pipermail/aptitude-devel/attachments/20130108/73f6948f/attachment.ksh>
-------------- next part --------------


-- System Information:
Debian Release: 7.0
  APT prefers unstable
  APT policy: (990, 'unstable'), (600, 'stable')
Architecture: i386 (i686)

Kernel: Linux 3.7-trunk-686-pae (SMP w/1 CPU core)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages aptitude depends on:
ii  aptitude-common           0.6.8.2-1
ii  libapt-pkg4.12            0.9.7.7
ii  libboost-iostreams1.49.0  1.49.0-3.1
ii  libc6                     2.13-38
ii  libcwidget3               0.5.16-3.4
ii  libept1.4.12              1.0.9
ii  libgcc1                   1:4.7.2-5
ii  libncursesw5              5.9-10
ii  libsigc++-2.0-0c2a        2.2.10-0.2
ii  libsqlite3-0              3.7.15.1-1
ii  libstdc++6                4.7.2-5
ii  libtinfo5                 5.9-10
ii  libxapian22               1.2.12-2
ii  zlib1g                    1:1.2.7.dfsg-13

Versions of packages aptitude recommends:
pn  apt-xapian-index                <none>
pn  aptitude-doc-en | aptitude-doc  <none>
ii  libparse-debianchangelog-perl   1.2.0-1
ii  sensible-utils                  0.0.7

Versions of packages aptitude suggests:
pn  debtags  <none>
pn  tasksel  <none>

-- no debconf information