[Freedombox-discuss] Dependency problem for matrix-synapse.

[Diederik de Haas]

First I want to make a few remarks, possibly for you but surely for other 
people who may be reading this (too).

1) It is perfectly fine to have no updates for days or even weeks on a Debian 
Stable system. FreedomBox is primarily based on Stable, but it also has 
stable-backports and therefor *may* have more frequent updates then a purely 
Stable system.

2) One of FreedomBox's goal is to take system maintenance out the hands of 
users, so that inexperienced users can (fully) use the system too.
Having said that, it is a Debian system which consists of FREE(dom) software, 
so you have every right to modify it as you see fit (as long as you respect the 
licenses). And you as the system administrator of FreedomBox ofc have that 
right too :-)
Experimenting (and breaking) things is a great way to learn more how the 
system works, which can be useful in several ways. And it can be fun.

I've 'shuffled' the previous mail/answers a bit as I hope that'll improve the 
learning experience.

On Wednesday, 12 January 2022 19:30:07 CET A. F. Cano wrote:
> On Wed, Jan 12, 2022 at 05:47:30PM +0100, Diederik de Haas wrote:
> > On woensdag 12 januari 2022 15:19:25 CET A. F. Cano wrote:
> It doesn't seem right that matrix-synapse-ldap3 is a recommendation (not
> a dependency) and yet...

In Debian a Dependency relation indicates that a piece of software can not, in 
any way, function (at all) without that other software package.

>From the description of matrix-synapse-ldap3:
"This package allows synapse to use LDAP for authentication as opposed to 
usual authentication via registering using a matrix client."

This shows that the package is not required and even the non-standard way, so 
surely not a dependency.
In other cases the link between 2 packages may be stronger, like you *really* 
want that other package installed too for most of the functionality (in the 
most used use-case), but *technically* you can use package A without package 
B, then that is a Recommended relation.
Quite a number of years ago it was different, but nowadays you REALLY should 
install Recommends. It's also the default.
If you don't, you should be capable of detecting and fixing things when they 
break, which will likely happen at some point.

Then there's a Suggests relation, which indicates that there's a (good?) 
chance you want package C too as it can be useful, but it is not required to 
use the software.
(There's also an Enhances relation, but I actually don't know in what sense 
that differs from Suggests)

What often causes confusion wrt Depends and Recommends is that something may 
feel like a dependency relation, but it technically is not.
It may even be a package which is needed for most use cases, but there is a 
use case where it is not, then it's still a Recommends.

Now let's examine your situation.

> > What may help is the output of the following commands:
> > - aptitude versions matrix-synapse
> 
> $ sudo aptitude versions matrix-synapse
>  i   1.48.0-1~bpo11+1                                                    100
> p   1.49.0-1~bpo11+2         bullseye-backports     100

The version you have installed is no longer known to any repositories you have 
configured on your system, while the one that is, is not (yet) installed.
Not catastrophic, but not good either. Let's find out why that is...

FTR: you can do many 'aptitude' commands without sudo. You only need sudo when 
you modify the system by installing or upgrading packages. But the 'versions', 
'why', 'search' and also 'install/upgrade -s' where the -s stands for 
simulate, can be done without sudo (and you therefor should).

> > - aptitude search ~b
> The -b option doesn't seem to be valid

It's not 'minus b', but 'tilde b' and it would return broken dependencies.
See https://www.debian.org/doc/manuals/aptitude/ch02s04s05.en.html for that 
and many more options you can use to query (APT through) aptitude.
If there are broken dependencies, that is bad and should be remedied asap.

> > > turns out it needed python3-authlib and python3-jwt.  Installed those
> > > manually and then tried again.
> $ sudo aptitude why python3-authlib
> i   matrix-synapse Suggests python3-authlib (>= 0.14.0)
> 
> > - aptitude why python3-jwt
> 
> $ sudo aptitude why python3-jwt
> i   matrix-synapse Suggests python3-jwt (>= 1.6.4)

This shows there is no Depends or Recommends relation, only a Suggests.
Why do you think they are needed (which I interpret as Depends/Recommends)?

I'll now explain something about package state, because it seems like the best 
place _sequentially_ in my response. But it shows the package state of the 
first shown package, which is matrix-synapse in this case, not python3-jwt.

Note that the 1st 'column' in aptitude's result gives useful info:
- 'i' means MANUALLY installed
- 'i A' means AUTOMATICALLY installed (as Depends or Recommends)
- 'p' means available but not installed.

The difference between manually and automatically installed is important for 
aptitude in finding possible solutions to an upgrade problem.
If a package was automatically installed, but it is no longer needed, then it 
can safely be removed (and aptitude (usually?) does that).
If a package is manually installed (by the system administrator, ie you), then 
you tell aptitude "I want to have this package" and aptitude respects that.
If there is an installation/upgrade issue to be resolved, then aptitude 
searches for possible solutions (this is one of the great strengths of 
aptitude over just apt(-get)) and solutions which would result in removing 
manually installed package get put at the bottom of the list as you 
(apparently) had a good reason to install that package.
If you 'push' aptitude long enough by rejecting (many) possible solutions (or 
explicitly directing towards such a solution), then it may 'at last resort' 
even offer that, but you'd normally not see those.

So if you don't have an *explicit* reason why you want/need python3-authlib 
and/or python3-jwt, I would suggest to change it (package) state to 
automatically installed, if they're not already.
This is changing the system so you do need sudo for this:
$ sudo aptitude markauto python3-authlib python3-jwt

If they were already marked as automatically installed, the above command will 
not actually do anything.

This likely will not remove those packages as there's still a Suggests 
relation, but it will give aptitude more freedom in finding possible solutions.

> > > It now needs python3-frozendict, which is already installed (at version
> > > 3.38.0-2).  It now breaks matrix-synapse-ldap3.  Removed it. 
> > > Reinstalled it. Actually 2 packages were re-installed, but it's back at
> > > "Breaks matrix-synapse-ldap3 (< 0.1.3-2-)".

The python3-frozendict package is an actual Depends of matrix-synapse so if 
you actually removed it, you very likely forced things (hard).
When you 'need' to force things, that likely means things are not in a very 
healthy 'state' and you'd make it (far) worse by forcing it. There is (very) 
likely a better solution, which _can_ also be to wait a few days.

While typing this response, I saw Sunil's mail come by. Having backports 
enabled, which FreedomBox does, can complicate things. But forcing things 
really should be considered a last-resort type of action.

> $ sudo aptitude search matrix-synapse
> i   matrix-synapse                  - Matrix reference homeserver
> i   matrix-synapse-ldap3      - LDAP auth provider for the Matrix homeser
> v python3-matrix-synapse-ldap3    -

I would mark matrix-synapse-ldap3 as automatically installed. It is useless by 
itself, without matrix-synapse, and it is Recommended by matrix-synapse, so it 
won't be removed. IMO it would improve the 'health' of the apt/aptitude 
package state database.

> > The 2 'why' queries are because those packages are *suggested* by matrix-
> > synapse, so I'd like to know what makes them needed.
> 
> That is the question, isn't it? Even with those packages installed,
> there is still the issue with matrix-synapse-ldap3

I have not yet found a relationship between python3-jwt/python3-authlib and 
matrix-synapse-ldap3, so for the time being I'll assume those are not 
connected.

> In interactive aptitude

That is often (also) called the TUI interface.
I will only give commands for the 'CLI' interface, to be executed directly in 
the terminal. Mostly because that's what I'm (quite) familiar with, but also 
because it's easy, and very useful in 'debugging', to copy the full response 
in the/a reply. The *full* response of aptitude tends to be quite informative.

> python3-frozendict is reported at 1.2-2, so it looks like this is the
> explanation: 1.4.9 matrix-synapse requires 1.2-3.
> Whether it is correct that this dependency prevents the upgrade to 1.49,
> I don't know.

Given Sunil's response, it looks like your conclusion is correct :)

> After deleting and undeleting (from within aptitude) frozendict is no longer
> marked as UNSATISFIED, the dependencies part only says:

Within aptitude's TUI you can 'mock about' quite a bit (IIRC), but if you'd 
have committed the 'deletion' of python3-frozendict, you would've broken your 
system.


> But matrix-synapse still breaks matrix-synapse-ldap3 and the 1.49
> version is still broken:
> 
>   --\ Versions of matrix-synapse (2)
> id   1.48.0-1~bpo11+1                   -8,164 kB
> pB   1.49.0-1~bpo11+2                   +8,288 kB

https://www.debian.org/doc/manuals/aptitude/ch02s02s02.en.html explains those 
state and action flags ... not good.

> I don't know what else to try.

While Sunil may already have found the (full) solution, if you've managed to 
read this ridiculously long reply this far ;-P I should also tell/explain how 
I would (try to) solve it. And this is exactly/actually why aptitude is so 
great (IMO): dependency resolving.

As mentioned earlier, I suggest to do this via the CLI interface and first try 
it with the '-s' parameter for 'simulate' and you can do that without sudo.

$ aptitude -s install matrix-synapse=1.49.0-1~bpo11+2

This will install/upgrade matrix-synapse to version 1.49.0-1~bpo11+2 and 
aptitude will search for a way to make that happen and suggest it to you.
It may be that you don't like its first suggestion and if so, you should 
respond with 'n' (No) and aptitude will search for another suggestion. You can 
keep doing that until you find a proper solution or aptitude (really) can't find 
any other solution to suggest to you.

With each suggestion, it will tell you (in detail) what/why that suggestion is 
'problematic'. If it is in no way problematic, it'll just install/upgrade the 
package, but that doesn't seem to be the case here.
You can share the output here and I can help you interpret what aptitude is 
telling you. But if you understand it enough, you should be able to fix it by 
yourself. You can say 'q' to any suggestion aptitude gives, to quit the 
problem resolver and fix the problem you found. And then you run the command 
again. You're in simulate-mode, so you can do whatever you want without 
breaking your system. So do feel free to experiment in this mode!

You can do that as many times as you like until you're satisfied with the 
result. And then you do the same command, but now without '-s' and with sudo 
to actually apply the solution you found earlier.

HTH,
  Diederik
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 228 bytes
Desc: This is a digitally signed message part.
URL: <http://alioth-lists.debian.net/pipermail/freedombox-discuss/attachments/20220113/d7fc1d6c/attachment.sig>