[Aptitude-devel] Bug#658635: Resolver extremely reluctant to upgrade packages to experimental, even as dependencies of packages in experimental

Sun Jun 12 00:05:34 UTC 2016

2016-06-11 19:06 Josh Triplett:
>On Sat, Jun 11, 2016 at 06:57:14PM +0100, Manuel A. Fernandez Montecelo wrote:
>> The mechanism is there, it's just not automatically applied just by
>> asking to upgrade, and I don't think that it should be changed at this
>> point.
>
>I don't think that installing a package version from experimental should
>change the target release; I'm just suggesting that it affect the
>heuristic weights for things like "should I upgrade a package to a
>non-default distribution".

Yes, I had understood your suggestion that it was only temporary.  I
just don't like that it's not very explicit (as in having session-wide
implications just for marking a single package from "experimental").

aptitude in the curses UI already upgrades happily a package to
experimental, as long as it doesn't need more packages upgraded.

The reason why I find the current behaviour useful is precisely what
annoys you -- that it highlights the fact that the normal/basic apt
resolution in the given pin-priority configuration doesn't like the idea
of upgrading a package to a non-default version when it needs other
packages upgraded to a non-default version, so one cannot avoid to
explicitly acknowledge and accept the result of *all* packages involved.

(Or temporarily and explicitly set Default-Release to
"experimental"/non-default-versions; or permanently set pin-priority of
the packages that one wants often from experimental at the same level).

In that way, there's no excuse for bug reports two weeks later saying "I
was wondering why libblah1 in my system is from experimental, and saw in
the log that it was upgraded to experimental 2 weeks ago and it didn't
give me any / enough notice at the time!!  aptitude is breaking my
system / making it insecure / whatever!1`11`!!  You've broken my system
and should do this and that so this never happens again!!1`1`!".

Even with the current config, we receive those quite often.  At least
now we can say "Well, aptitude told you about that".

If instead, as you propose, aptitude accepts that by asking pkgA to be
upgraded to experimental, all packages that are needed to solve the
dependencies/conflicts are upgraded to experimental without much / any
fuss, and one carelessly presses "g" twice (or doesn't review carefully
because one didn't see any "scary" resolver conflicts), the installation
can proceed without much notice, and then people realise weeks/months
later that has a version of some package that one didn't request
explicitly for experimental, and thus blame aptitude for that.

In this regard, "experimental" is specially problematic because
sometimes packages are uploaded as a... well, experiment, get out of
date and are broken or have no security support or normal updates
whatsoever, and preclude the lower versions from unstable (which are
updated frequently, normally) to be installed in the given system, even
long after the version of the package from experimental might not be
needed (perhaps because the user removed the package that depended on
that specific version).

So this is the situation that I would like to avoid, and for which
there's no substitute of the user explicitly acknowledging the situation
(either by resolver actions, setting Default-Release or permanent
pin-priority changes).

>> In the command-line execution, the view is something like this:
>>
>>   =========================
>>   The following packages will be upgraded:
>>     libreoffice-base-core libreoffice-calc libreoffice-common libreoffice-core
>>     libreoffice-draw libreoffice-impress libreoffice-kde libreoffice-math
>>     libreoffice-style-breeze libreoffice-style-galaxy libreoffice-style-oxygen
>>     libreoffice-style-tango libreoffice-writer python3-uno uno-libs3 ure
>>   =========================
>>
>> If the resolver is not invoked, versions are not shown by default, so
>> users cannot see if they are upgrading to versions of experimental
>> until the download+installation takes place.
>
>That seems potentially fixable.  What about "The following packages will
>be upgraded to experimental:", in a separate heading?  (Likewise for any
>packages upgraded to a non-default distribution.)
>
>Or, alternatively, add the "/experimental" suffixes, but that seems
>noisier; a separate heading reduces the noise.

That would be one way to make it more explicit, yes.  I would prefer the
first.

Still, there's the issue of the "explicitness".

>> >> (The fact that it suggested "remove" as a first action was a well-known
>> >> problems present for many years, independently of this particular case).
>> >>
>> >> Other than that, aptitude's resolver is a complex beast which works
>> >> based on heuristics.  The way to guide those heuristics is to give
>> >> user-desired hints.  It will suggest to Keep few packages installed
>> >> rather than upgrade many, specially if they come from non-default
>> >> versions.
>> >
>> > I don't think it suffices to just wave off sub-optimal results from the
>> > resolver with "the resolver is subtle and quick to anger, and your
>> > packages are crunchy and taste good with ketchup".  Yes, the resolver
>> > won't be perfect every time, but it could potentially be improved.
>>
>> That aptitude's resolver is quick to anger is one way to view it.
>
>It was a joke, not an assessment.  The resolver is awesome and helpful,
>just not perfect.
>
>> I've been improving the resolver lately in other ways, I just don't
>> see this as a useful improvement, sorry.
>
>Sad to hear that; this remains one of the few issues I still run into
>regularly with aptitude.

One man's issue is another man's feature :-)

Apart from avoiding some class of bug reports related with this, I
myself like to see and acknowledge explicitly the conflicts when they
happen, so I am alert in the future about possible problems.

To be honest, from my point of view, I think that either explicitly
setting the Default-Release in such cases, or permanently pinning
iceweasel/firefox and related packages, is a good solution and not too
annoying.

The specific problem with tweaking the resolver is that there are no
hard rules, everything is scores of given actions, exploring a graph of
solutions semi-randomly, and balances.

For example, I spent a good few weeks trying to solve the problems that
crept up in the last few years, solved in 0.8, trying to make that
"upgrades" are offered more often and in preference of "keeps" or
"removals".  Still, this doesn't work in all cases because removals are
necessary for obsolete packages and transitions, so one cannot forbid
them, and it's difficult to balance.

There are other problems.  For example in the past, exploring more steps
of the graph was encouraged and the overall score of more actions was
offsetting the negativity of removals, so sometimes aptitude suggested
to remove more packages than necessary; or suggested 30 removals before
1 upgrade or 2 keeps.  By changing that the solutions are more
"sensible", but takes significantly longer to reach some results,
possibly too much long in some cases (transitions or dist-upgrades),
perhaps making such cases to take hours, and thus rendering aptitude
useless for those cases.

The relation of all this with the non-default versions / experimental is
that, if I change the resolver to assign equal value to experimental
versions the first time that the user selects a version of some package
from experimental, it can easily suggest to upgrade more packages to
experimental than it's strictly necessary (similar case as the
offsetting of removals explained above), just because the score happens
to be slightly higher (packages in experimental with less dependencies
overall than the current ones in unstable, or packages in experimental
upgraded to "Priority: important" which scores higher, etc).

And in your example, you just want to upgrade iceweasel to experimental
isolatedly and do nothing else in that session; but in practice many /
most user sessions consist in a mix of many actions, and by selecting
versions from experimental and changing the scores in all the generated
graph as necessary, perhaps other unrelated packages selected later in
the same session would also be chosen from experimental, when they
wouldn't have been if this priority bump hadn't happened.

So a session upgrading iceweasel + pkgA could make that pkgA is chosen
from experimental, while if one had upgraded pkgA isolatedly (or with
other packages, but all from unstable), pkgA would be upgraded to
unstable only.

... So, in summary, it's a quite complex problem with many many *many*
ramifications.  And as I said, having all things into account, I think
that explicitly setting the Default-Release, permanently pinning or
guiding the resolver is not very high price to pay when weighing
pros-and-cons of the changes that you request.  Sorry.

Cheers.
-- 
Manuel A. Fernandez Montecelo <manuel.montezelo at gmail.com>