[Aptitude-devel] Bug#782777: actions on specific packages don't work on single-arch systems

Wed Apr 22 11:18:28 UTC 2015

Control: reassign -1 apt 1.0.9.8
Control: affects -1 aptitude
Control: severity -1 important
Control: tags -1 patch
Control: retitle -1 actions on specific packages don't work on single-arch systems

Hi,

(the mail starts with a LOT of technical bla bla a user doesn't need to
 concern itself with. I have cc'ed all reporters explicitly through as
 I would like them to try the "workarounds" mentioned further below, so
 feel free to skip over the technical reasoning straight to the 'meat'.
 And if you gonna reply, please ONLY to the bugreport, thanks!)

On Tue, Apr 21, 2015 at 08:17:36PM +0200, Axel Beckert wrote:
> David already seems to on to something...

After quite some hair-pulling and the occasional swearing which Axel
thankfully endured I finally arrived at a rather misfortune conclusion
(at least for me): I was wrong, which is quite a hurtful experience as
I had put quite a bit of energy into preventing it…

So, desperate me claiming the fix for #777760 would have a low
regression risk as there aren't that many packages in existence which
expect this behaviour, I forgot to consider a potential side-effect:

The correct creation of a package foo from architecture alien1 can if we
haven't seen the package foo from our native architecture yet claim the
price of being the first known package foo. The first place is assumed
to belong to the package of the native architecture through, so a libapt
application asking for the package foo of native architecture will get
unconditionally handed the first package foo found, which if we are
unlucky is the one for alien1 and not native…

This assumption used to be true, but with the advent of architecture
specific dependencies (and the deprecated policy-forbidden packages
without architecture) not so much anymore. It is to be noted that not
all architecture specific dependencies actually trigger this, they just
have the potential to do it, if they do depends very much on the order
of things, but we will get to that in a minute.

Analysing amd64 sid+experimental you can find the following effected:
libstdc++6-4.9-dbg arm64
libgcc-4.9-dev arm64
libgcc1-dbg arm64
libgomp1-dbg arm64
libitm1-dbg arm64
libatomic1-dbg arm64
libasan1-dbg arm64
liblsan0-dbg arm64
libtsan0-dbg arm64
libubsan0-dbg arm64
libcilkrts5-dbg arm64
libquadmath-dbg arm64
libgfortran-4.9-dev arm64
libgfortran3-dbg arm64
u-boot armhf

(the second column is the architecture the first package is created for,
which should have been amd64 if the made assumption would be valid).

We have seen the names of most of them in this bugreport already, Axel used
libgcc1-dbg in his example.

The situation is a bit different for jessie sources:
Only u-boot is effected here. u-boot:amd64 doesn't exist btw.
(The reason is that crossbuild-* doesn't exist there).

Note that this error doesn't add up, but can decrease with additional
sources. Having testing+sid+experimental enabled e.g. leaves you with
only two of them still on the list (if you have it in this exact order).
Architectures aren't created equally either, but they are all playing in
the same ballpark, so list isn't changing much if at all between archs.

Bug #782889 mentions other packages effected on jessie through, and Axel
has an example with libgnutls26 on an amd64 with sid+experimental, so
how does these happen?

It took me longer to realize than it should have, but it is actually
quite simple now that I had a good 'talk' with my pillow:
In my testing I was using either only the component "main" or the
components "main contrib non-free" (in that order), but if you happen to
do (lets say) "contrib main" you end up with these in addition:
libldap-2.4-2 i386
libdb5.3 i386
libgnutls26 i386
libsasl2-2 i386
libsasl2-modules i386
libsasl2-modules-db i386

Which minus the libgnutls26 are also exactly the packages #782889
mentions for jessie (libgnutls26 isn't existing nor even referenced in
this release, so the package comes never into existence for apt here).
[and they are the only packages mentioned by the bugreport, so not all
 jessie users are effected at all – see also "workaround" 1.]

So, what is the effect you might ask now:
There are more or less four ways to get to a package in libapt.
1. You can follow a dependency. There is no lookup involved here, this
will always point to the correct package as this one is prebuild with 2.
2. You get a Grp-structure, which has a FindPkg(arch) method giving you
the package you desire.
3. You iterate over ALL packages one after the other (that sounds
ridiculous stupid, but is done a lot as you usually have to touch them
all in some way anyway)
4. You are asking the cache directly with a FindPkg(name[, arch]) call
for a package.

Only the last one is effected as this path includes a detection for
single-arch which then has the incorrect assumption of the first package
being the native package. The attached patch removes the single-arch
special casing in this codepath, which means that the same code as on
multi-arch is used which will in turn use 2. to get the package.

apt itself it turns out uses way 4 very seldom as its dealing a lot
with Grps nowadays and hence does 2 (if not the other two), so, even
through the output in simulation is slightly off you can interact with
these packages quite normally (Axel e.g. showed already that he can
install such a package). Upgrades and such will work, too.

aptitude and other frontends potentially use 4 more as most of them
weren't changed much since the advent of Multi-Arch (and 2 is a thing
only since Multi-Arch), so I suspect interactive actions like in
aptitude the marking where one of these packages is picked specifically
to be effected, while non-interactive actions like upgrades where the
package is picked via dependencies (1) or whole system operations (3)
aren't.

The worst thing which can happen is that effected packages can't be
installed with (e.g.) aptitude, not forgetting new or forgetting auto
(also with apt) while being slightly annoying isn't dangerous. I want to
highlight especially that it isn't possible to produce a broken system
with this as dependencies are created and evaluated correctly.

---- [ end of (most of) the techno-babble ] ----

Effected users have a cornucopia of options to deal with this, which
brings in other small benefits as well, so users might want to try:

1. Reorder components in the sources.list if you have activated
multiple, which means if you currently have e.g.:
deb http://http.debian.net/debian/ jessie contrib non-free main
change it to
deb http://http.debian.net/debian/ jessie main contrib non-free

For jessie users this will trim down the list to 'u-boot' mentioned
above, which as said has no native counterpart, and is therefore handled
(= ignored) correctly already, so this should make this "problem" go
away entirely already for jessie users without any downside.
(and is hence the reason why I am not crying RC-BUG and pull the
 emergency brake just yet. We can fix that in a point-release.)

2. Reorder sources.list lines:
man sources.list defines that the order of these lines effect from which
source a package is pulled if multiple provide the same package version.
Using here a pattern of (omitting the ones you don't want of course)
oldstable
stable
testing
unstable
experimental
security, -updates, -backports, third-parties, …
instead of the other way around allows for e.g. download progress output
to be better as it will report that a package which already migrated from
unstable to testing is downloaded from testing, rather than claiming it
comes from unstable (which can be confusing if you happen to have pinned
down unstable, but your package manager seems to install packages from
it anyhow… Its also good e.g. for the security mirrors to avoid
downloading from them if your usual mirror has the update already so
they can serve others in need instead).

This should help at least a bit in reducing the list as the cache is
filled with most packages already before apt encounters the rather new
architecture-specific dependencies in newer releases (see my example
above - the remaining two were our friend u-boot and libquadmath-dbg,
which is another non-existing package, so problem gone also here,
which explains why some systems with the 'same' sources are effected
while others aren't; order matters quite a bit sometimes…)

3. Ignore it for the time being
Especially if you have opted for the two previous options, the still
effected packages list should be small if not empty and include mostly
packages most users do not care too much about in the first place (*-dev
and *-dbg). So if you don't have a pressing need, an itchy trigger
finger or aren't on an at least immediate Debian experience level
sitting that one out until a point release fixes this is a very valid
option instead of rushing further down the list.

4. Enable Multi-Arch
This makes the problem go away entirely, but with all the advantages and
disadvantages Multi-Arch itself might have for you.
(In effect, it triggers the codepaths being used as the patch does)

5. Adventure seekers can try the attached patch. Axel already tried it
out for me and confirmed it fixes things and my tests are reassuring as
well, but additional feedback never hurts (still, this is for the
experienced Debian user to try only!)

A "thank you" goes out to everyone reporting this bug and especially to
Axel for helping me debug this – and of course a "sorry" to everyone
for breaking things, especially to the release team.

Best regards

David Kalnischkies
-------------- next part --------------
A non-text attachment was scrubbed...
Name: apt-782777-single-arch-weirdness.patch
Type: text/x-diff
Size: 6528 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/aptitude-devel/attachments/20150422/ae72953e/attachment-0001.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/aptitude-devel/attachments/20150422/ae72953e/attachment-0001.sig>