[pkg-bacula-devel] Bug#1109499: bacula-director-sqlite3: fails to dist-upgrade from bookworm to trixie

[Carsten Leonhardt]

Hi,

Adrian Bunk <bunk at debian.org> writes:

> On Fri, Aug 01, 2025 at 07:57:37PM +0200, Carsten Leonhardt wrote:
>> Hi,
>
> Hi Carsten,
>
>>...
>> I'm not a database expert, but my guess is that SQL statements do not
>> result in atomic operations on disk.
>
> a core feature of SQL databases is ACID (atomicity, consistency, 
> isolation, durability).
>
> And this is not limited to single SQL statements:
>
> You can write any number of SQL statements between BEGIN and COMMIT,
> and the database ensures that either all of them or none of them happen.
>
> You can put a billion SQL statements in one transaction, and the database
> ensures that at any point in time either all or none of them are visible.
>
> This includes what is visible to other database users accessing the 
> database at the same time.

This sounds well in theory. In practice I read:

> Roll back of incomplete transactions

> Incomplete transactions are any transactions that were active at the
> time of unexpected exit or fast shutdown. The time it takes to roll
> back an incomplete transaction can be three or four times the amount
> of time a transaction is active before it is interrupted, depending on
> server load.

> You cannot cancel transactions that are being rolled back. In extreme
> cases, when rolling back transactions is expected to take an
> exceptionally long time, it may be faster to start InnoDB with an
> innodb_force_recovery setting of 3 or greater. See Section 17.20.3,
> “Forcing InnoDB Recovery”.

Source: https://dev.mysql.com/doc/refman/8.4/en/innodb-recovery.html#innodb-crash-recovery

So if the database operations done during the upgrade of bacula fail
near the end (possibly due to the database server running out of space),
it may take 3-4 days just to roll back in the case of Sven's database
that I referred to earlier.

Another quote from MySQL: "MySQL includes components such as the InnoDB
storage engine that adhere closely to the ACID model [...]" (Source:
https://dev.mysql.com/doc/refman/8.4/en/mysql-acid.html)

To me (and others) this reads as "only InnoDB adheres to the ACID
model", the other database engines do not.

(For MariaDB I can only find that they list the ACID principles without
further details on their implementation)

>> If the server runs out of space,
>> loses power or whatever, the database will most probably be in an
>> undefined state. I wouldn't know how to automatically recover and
>> continue database operations from the point where it stopped.
>
> In all the error scenarios you describe, the well-defined state of an 
> SQL database is the state without the transaction.
>
> I haven't checked what exactly Bacula does during a schema upgrade,
> but what you write sounds as if the problem might not even exist.

For MariaDB/MySQL I've been in the situation myself that I couldn't
cleanly recover from a full disk. It's not always possible to clear up
or add space while the machine is running.

For PostgreSQL: "If you let the disk fill up, you can't recover from within PostgreSQL."
(Source: https://dba.stackexchange.com/questions/167515/dealing-with-disk-space-full-in-postgresql)

>> > Do you really want to receive the Release Notes in the form of
>> > 100 debconf questions you have to manually acknowledge on every
>> > machine you upgrade?
>> >
>> > Users will just be annoyed and click through it without reading,
>> > as is already common on the web.
>> >
>> > The user is supposed to read the Release Notes before upgrading,
>> > if the user chooses not to read them that's not our problem.
>> 
>> As you can guess, I would appreciate getting warned about performing an
>> operation that could possibly result in an outage of services on servers
>> without a direct relation to the server I'm upgrading: There will be
>> users with a dedicated database server. They can use it for more
>> applications than just bacula. Even having a redundant database server
>> setup wouldn't help.
>> 
>> Sure, we can choose to say it's the user's fault for not reading and
>> remembering the release notes.
>> 
>> But my opinion is still to err on the safe side: abort while it's
>> safe(r) to do so rather than running into an error condition that can be
>> magnitudes harder to resolve, especially while your backup is out of
>> order.
>
> You are not erring on the safe side, you are making debconf unusable.
>
> You want to click through hundreds of debconf confirmation messages on 
> every machine you are upgrading?
>
> Every maintainer with a package that has a NEWS.Debian might argue the 
> same way you do, and additionally add a debconf confirmation.
>
> You seem to think that among the ten thousands of packages in Debian 
> your package is the one that is so super-important that it has to force 
> its NEWS.Debian through debconf even on users who only want to see 
> critical debconf quesions.
>
> Depending on how you upgrade and how many machines you upgrade debconf 
> can be quite annoying, the user either has to manually click through 
> everything or preseed all answers.
>
> A typical upgrade might upgrade several thousand packages.
>
> debconf questions that are not necessary are a PITA for users,
> don't punish everyone for users who did not read the Release Notes.

I understand you don't like debconf popping up and probably see this
instance as possibly opening a floodgate.

I think I've explained why I think that in this instance it is important
that the users know what they are getting into. It's not about "my
package is exceptionally important" but rather the consequences of
failing to understand that what is about to start can lead to situations
that can take days to recover from. Mind you, the debconf pops up only
when a bacula-director is installed, which is usually only one machine
per site. Popcon reports around 200 installed bacula-director packages,
so the "audience" is rather small.

We obviously have a different opinion on how hard we want to punish
users that don't read and remember the release notes. I'm not aware of a
project consensus about how to treat these users.


Lastly, I'll note that a version of bacula with the database upgrade
entered Ubuntu with it's 24.04 release, followed by a version with the
debconf confirmation in the 24.10 release. I don't see bug reports
there, neither about a failed upgrade nor about the debonf confirmation
being unwelcome. My guess is that people with larger installations stick
to LTS releases when running bacula-director on Ubuntu.

Regards

Carsten