[Soc-coordination] GSoC - APT Ordering - Report 2
cbaines8 at gmail.com
Fri Jun 17 18:34:33 UTC 2011
Since my last report , not a lot has changed with respect to my
project, this is because all last week I have had exams and have not
been able to work. However due to the tendency of the mind to wonder
when attempting to revise colour changes in ligand substitution
reactions and the equations for the scalar product of vectors, I have
been thinking about my project and the apt code in general.
One thing that I have done since my last report is some further
investigation in to the ordering failure Michael showed me. I have
included some of the output from my testing (… are lines removed for
I will apologise to anyone in advance if the following attempt at code
annotations goes wrong.
SmartConfigure openoffice.org-common ← SmartConfigure is a method
used in APT to perform
configuration on a package and
all of its dependants.
DepAdd: openoffice.org-common ← DepAdd recursively (it calls itself)
adds all the dependants of the package
to the configuration list. You can see
the iteration as it quite nicely
indents the output.
DepAdd: libcurl3-gnutls ← Here is one of the packages I am
interested in, its been added to the
configure list, the DepAdd routine is
now looking at its dependencies.
DepAdd: libldap-2.4-2 ← Its found the other package of
interest, as you can see
libcurl3-gnutls depends on
libldap-2.4-2, this means that
libldap-2.4-2 must be configured
DepAdd FAILS on: openoffice.org-core ← Something has gone wrong
DepAdd FAILS on: openoffice.org-style-oxygen
Conf libsysfs2 ... ← But even thought there has been a error, the
configuration of the list begins
Conf libcurl3-gnutls ... ← Now here is the problem, not only is
libcurl3-gnutls being configured before
libldap-2.4-2, libldap-2.4-2 is not even
configured with this group of packages!
Conf libxml2 (2.7.6.dfsg-1ubuntu1.1 Ubuntu:10.04/lucid-updates [amd64])
Something must have happened, or not happened in the Dep add routine to
cause the routine to ignore libldap-2.4-2. At the moment I have not had
the opportunity to discuss this with my Mentor, this is something I
will hopefully learn more about and do further investigation in to the
code this coming week.
Currently I have written a small piece of code to intercept configure
calls from the package manager in APT and check for any unconfigured
dependencies (This is currently not in any repository). This currently
correctly detects the error above, but as the following example shows it
does not cope with dependency loops.
Here is the relevant output from APT:
Conf firefox (3.6.17+build3+nobinonly-0ubuntu0.10.04.1
Conf firefox-branding (3.6.17+build3+nobinonly-0ubuntu0.10.04.1
E: Package firefox should not be configured because package
firefox-branding is not configured
Now firefox depends on firefox-branding (seems sensible enough), but
firefox-branding also depends on firefox (this does not seem as
sensible). This results in a dependency loop. To configure these
packages, the loop needs to be broken, the Debian policy manual covers
the exact mechanics of how to do this , but as far as apt is
concerned, dpkg will handle this.
The way this works is the Conf statements that appear in the output
above are grouped together by APT, and inserted in to one call to dpkg.
So the dpkg invocation will look like:
dpkg --configure A B C
dpkg --unpack D
dpkg --configure D
dpkg --configure A
dpkg --configure B
dpkg --configure C
dpkg --unpack D
dpkg --configure D
Thanks go to David Kalnischkies for the above example.
So the initial outcome of this is that my code is detecting these
breakages as errors where they are actually none. So I could add to my
check to see if there is a dependency loop between the two packages
concerned, but while this is simple enough for the example above there
could be any number of packages involved in this loop, so I am not sure
if this is the best approach.
There is also another possible problem though, as I said above APT
passes packages to dpkg in batches, now if there is a large run of
packages being configured (which does happen) this call will get split
in to many (because of limitations outside APT's control).
Now imagine what would happen if for the above example the large list of
packages was split between firefox and firefox-branding, dpkg would not
be able configure the packages properly. Now this might or might not
impact firefox (depending on the way the package works), but for some
packages this could be problematic.
One solution I can think of for this problem, is before ordering
packages, split the list of packages in to batches (groups of packages).
Now this has two advantages:
- By limiting the size of batches, you can prevent any splitting of
dpkg calls (without doing any complex analysis of dependencies for a
- Batches could be processed in parallel, allowing packages to be
installed and fetched at the same time.
There are some practical considerations about the process needed for
separating packages in to batches e.g. criteria for separating
packages and how do deal with dependency loops and large batches, but
from thinking about the problems at hand, this seems to me to be a good
and flexible approach.
I am now just days from my final exam (on Tuesday), and from then I will
be free to work on APT full time (which I am very much looking forward
Also if anyone has any advice, comments, questions or corrections, feel
free to voice them.
http://lists.alioth.debian.org/pipermail/soc-coordination/2011-June/001005.html | http://lists.debian.org/deity/2011/06/msg00005.html
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 836 bytes
Desc: This is a digitally signed message part
More information about the Soc-coordination