[Aptitude-devel] Bug#988220: aptitude: documentation improvements

[Christoph Anton Mitterer]

Source: aptitude
Version: 0.8.13-3
Severity: wishlist


Hey.

I was looking through the documentation, specifically the part about search strings
and may have found some ambiguities or things that would benefit from more
clarification:


1) Searching for strings
Mentiones that most patterns are taken as regular expressions, but it seems it's
nowhere mentioned which (POSIX BRE, ERE, PCRE or something else?) are actually used?




2) There's the paragraph starting with "The characters “!” and “|” have special
meanings within search patterns."

That's IMO a bit confusing:
- The whole paragraph is right in the chapter that talks about the "literal" text
  strings within patterns - so it doesn't feel as if the the ! and | here would
  mean the short-form-versions of ?not() and ?or().
- Of course there's also the | of the regexp itself.
  For the example given, ?description(grand~|oblique), it does not seem that
  ?description() allows another subpattern (like ?description(?or(foo, bar)),
  so one would guess the | and ! here would *not* mean ?or() and ?not?
- OTOH, it doesn't seem they mean the regexp patterns.

So what exactly is it and where can it be used?




3) In "Shorthand for search terms"
"Question marks (“?”) will not end the short form of a term,"
I would add here e.g. "(unlike "~", which will end the term)", as the example
with "~i~napt" above seems to apply.




4) Searches and versions
TBH, I have some difficulties to understand how this *exactly* works.
It kinda seems to miss the "great picture", how the matching is actually performed.

Does it go through the list of all package names (without versions) and
match each of them *then with versions* against the patterns and sub-patterns,
where some of the terms then take any/all versions?

Or is it rather like SQL and joins, that each (sub)pattern produces a list of
(package,version) tuples that are then intersected or unified?

At least I'd suspect it does not go through each single (package,version) tuple
and match (just that) against the pattern, cause that wouldn't work.


Like in the example: ?depends(?version(2\.0))
Is it more like:
a) From inner to outer:
   ?version(2\.0) returns all (package,version) tuples that contain "2.0", and
   ?depends() on that, returns all packages depending on any package of that?
or:
b) From outer to inner:
   Go through the list of all (packages,version) or just (package), for each
   of that:
   add it to the result if it depends on the pattern inside (?version(..))
   and then... well no idea?
   
   The explanation of that example seems to imply that there's some context
   switch: the outer ?depends() works on the "current package" foo,
   while the inner ?version(2\.0) works on the version of foo's dependencies.
   However, that doesn't seem to follow from
   /usr/share/doc/aptitude/html/en/ch02s04s05.html#searchDep


(a) would be more like working with joins/intersections/etc. on (multiple)
sets of (package,tuple).
(b) is more like how some parts of the documentation seem to explain it, but
kinda doesn't really seem to fit.

So I kinda miss the general way of how things are acutally processed - in which
direction and what's actually returned (packages? packages+versions?)

But that seems important for cases like:
?installed ?version(CANDIDATE)
=> when ?installed and ?version returns versions and implicit ?and matches only names, 
   this should yield something different if it also matches version (in which case
   it would, I guess, only match packages whose installed version equals the candidate
   version?



There seem to be quite some ambiguities
E.g. it says:
- "By default, a pattern matches a package if any version of the package matches the pattern."
  => sounds as if it would be only about the package name (not the version)
  and
  "For instance, the search term ?depends(pattern) will select any package that depends on a
   package matching pattern. However, pattern will only be checked against the versions of
  the package that actually satisfy a dependency."
  => That again sounds, as if the patterns would be evaluated from outer terms to inner terms.
  => What does a term actually "return"? Is it just the name of the package or is it one or
     many (package,version) tuples?

Or:
- "There is a subtle, but important, distinction between matching a pattern against a package,
   and matching it against all the versions of that package. When a pattern is matched against
   a package, each of its terms is matched against the package, and so each term will match if
   any version of the package matches."
  => The whole paragraph is rather difficult to understand.
     Seems to imply, that there are two kinds of patterns, one matching names the other
     matching (name,verson).
     Also I guess the crucial point is missing, namely that in the former case, the different
     (sub) terms of a pattern need not match against the same (of any) version?
but:
- the description of ?any-version(pattern) says:
  "To be precise, as with any other pattern, it is not packages but versions of the packages which
   are matched."
  => rather implyin that all terms woul be handled the same ann always match versions


Or:
The description of ?any-version(pattern) again:
"Matches a package if any one of its versions matches the enclosed pattern."
AFAIU, the real point here is, that it matches if the pattern (i.e. all sub-patterns) matches,
if all of them match against *the same version* of at least one of all the package's versions.


For ?archive() and ?origin, it would be nice if this would perhaps remind people, that
obsolete packages are obviously never matched.
Perhaps even, that archive is the same thing as "suite"?


Perhaps clarify, whether ?installed is exactly the same as ?version(CURRENT)?


IIRC, there were examples using things like this ?archive(testing), which is however dangerous,
as peoeple will adapt that to write ?archive(stable), which does however include unstable as well,
doesn't it?
I think all such patterns (also ?origin) should be rewritten like ^testing$



When ~narrow(pat1, pat2) is identical to ?any-version(pat1 pat2), is there any difference
between ?any-version(pat1 pat2 pat3) and ~narrow(pat1 pat3, pat2) or ~narrow(pat1, pat2 pat3)?


Does the "narrowed" context of ~narrow(pat1, pat2) only affect when matching pat1 with pat2
ot does it inherit down, e.g. consider:
aptitude search '?narrow(?installed, ?not( ?archive("^stable$") ))'
=> "easy"... AFAIU... the ?installed selects any (package,version) that is installed, and the
   intersects it with any (package,version) that is not in stable.

aptitude search '?narrow(?installed, ?not( ?archive("^stable$")  ?archive("^testing$") ))'
=> if the narrowed context goes down inside the ?not, then ?archive("^stable$")  ?archive("^testing$")
   should be empty, I guess, unless there are same versions in both stable and testing
   if it doesn't go down, then ?archive("^stable$")  ?archive("^testing$") should, AFAIU, yield
   all packages which have *any* (same/different) version in both, stable and testing.





5) Useful examples
I think it would be nice if the documentation gives useful examples.

Packages where the installed version comes from a certain origin:
  ?narrow( ?installed, ?origin("^Debian Backports$") )
with several possible origins:
  ?narrow( ?installed, ?origin("^(Debian Backports|My Local Organisation)$") )
  which I guess is effectively the same than:
  ?narrow( ?installed, ?origin("^Debian Backports$") | ?origin("^My Local Organisation$") )
  the former workgin with regex alternatives the later with pattern alternatives

=> these won't include obsolet packages, I guess


Packages where the installed version *does not* come from some origin (typically to
search for packages not from Debian):
  ?narrow( ?installed, !?origin("^Debian$") )
with several allowed origins:
  ?narrow( ?installed, !?origin("^(Debian|Debian Backports)$") )
  which I guess is effectively the same than:
  ?narrow( ?installed, !( ?origin("^Debian$") | ?origin("^Debian Backports$") ) )

=> these will also include obsolete packages and also packages which are actually (i.e. their
   "newest version) still in Debian, but just not yet upgraded
   so the query is in practise less useful, IMO


Packages where the installed version *does not* come from some suite (typically to
search for packages outside of e.g. stable (and perhaps the corresponding stable-updates
suite):
  ?narrow( ?installed, !?archive("^stable$") )
with several allowed archives:
  ?narrow( ?installed, !?archive("^(stable|stable-updates)$") )
  which I guess is effectively the same than:
  ?narrow( ?installed, !( ?archive("^Debian$") | ?origin("^stable-updates$") ) )

=> these will also include obsolete packages and also packages which are actually (i.e. their
   "newest version) still in Debian, but just not yet upgraded
   so the query is in practise less useful, IMO




So I guess what people really want in practise with the examples above, but using the
candidate instead of the installed version.
E.g. the following doesn't work:
  ?narrow( ?version(CANDIDATE), !?origin("^(Debian|Debian Backports)$") )
as it matches all candidate versions (of all, including not installed, packages).


I guess the following does the job, but I'm not totally sure, given that I don't
fully understand the matching yet:


Packages where the candidate version of installed packages *does not* come from some origin (typically to
search for packages not from Debian):
  ?installed ?narrow( ?version(CANDIDATE), !?origin("^Debian$") )
with several allowed origins:
  ?installed ?narrow( ?version(CANDIDATE), !?origin("^(Debian|Debian Backports)$") )

=> should still include obsolete packages


Packages where the candidate version of installed packages *does not* come from some suite (typically to
search for packages outside of e.g. stable (and perhaps the corresponding stable-updates
suite):
  ?installed ?narrow( ?version(CANDIDATE), !?archive("^stable$") )
with several allowed archives:
  ?installed ?narrow( ?version(CANDIDATE), !?archive("^(stable|stable-updates)$") )

=> should still include obsolete packages


The idea of these is, that the ?narrow(...) matches any candidate version, which are not
in the respecitve origin or suite, which would however include also non-installed packages.
So the "and" with ?installed filters out the later.


One should perhaps also mention, that the above is smart enough (AFAIU) to also select
packages which are e.g. still in some origin, but nevertheless won't get updates anymore:
I found the following case on one of our systems:
# aptitude --disable-columns --display-format '%c%M %p' search '?installed ?narrow( ?version(CANDIDATE), !?origin("^(Debian|Debian Backports)$") )'  |  column -t -s ' 'i
iA  libssl1.0.0

# apt-cache policy libssl1.0.0
libssl1.0.0:
  Installed: 1.0.1t-1+deb8u6
  Candidate: 1.0.1t-1+deb8u6
  Version table:
 *** 1.0.1t-1+deb8u6 100
        100 /var/lib/dpkg/status
     1.0.1t-1+deb7u4 600
         -1 http://security.debian.org/debian-security wheezy/updates/main amd64 Packages
     1.0.1e-2+deb7u20 600
         -1 http://debian.mirror.lrz.de/debian wheezy/main amd64 Packages

So while the package itself is still in some of the selecte origins (the two repos below),
the candidate version is not.
So libssl1.0.0 1.0.1t-1+deb8u6 wouldn't get any updates at this point and can still be considered
"dead".

I guess the same would also work in the archive case,... cause this is probably what admins
are really interested in:
  Do I have any packages for which I wouldn't get any updates anymore?

Not sure whether the examples above solve this also when taking priorities from apt_preferences
into account.


Maybe, for teaching it would be nice, if one could explain what's the difference from
  ?installed ?narrow(            ?version(CANDIDATE),            !?origin("^Debian$") )
to e.g.
             ?narrow( ?installed ?version(CANDIDATE),            !?origin("^Debian$") )
             ?narrow(            ?version(CANDIDATE), ?installed !?origin("^Debian$") )
AFAIU, the later two are identical? But they both would only match packages, where
the installed and candidate version are the same?




Well,... thanks for considering these ideas :-D

Cheers,
Chris.