[pkg-cryptsetup-devel] Bug#952703: [cryptsetup] needs breaks on old cryptsetup-bin

[jnqnfe at gmail.com]

On Fri, 2020-02-28 at 02:43 +0100, Guilhem Moulin wrote:
> On Thu, 27 Feb 2020 at 23:55:49 +0000, jnqnfe at gmail.com wrote:
> > It is thus not a question of 'FrankenDebian' but of how old a sid
> > installation upgrading is supported for.
> 
> The starting baseline doesn't matter, if you start from sid and don't
> upgrade for longer than a full recycle, then most of your packages
> are
> likely to be as old as the ones in oldstable.  Mixing stable and sid
> packages is asking for trouble, but at least the upgrade path from
> stable to sid is tested (the package in sid will be transitioning to
> testing, which is stable +1); upgrading from *old*stable to
> testing/sid
> is not, regardless whether if it's a selective or full upgrade.
> 
> > I don't think too much focus should be paid to the fact of just how
> > old
> > the version is in this case. Surely this affects all sid/testing
> > installations that have not been upgraded since the reorganisation
> > of
> > cryptsetup began (~May 2019), and became an issue at some point
> > towards
> > the end of the reorganisation, which nobody encountered or bothered
> > to
> > report until now.
> 
> We started the package split in *June 2018* with 2:2.0.3-1, and ended
> July

Oh, Okay. I recalled it being much sooner than that, time flies :)
edit: ah, so I see there were two reorganisations, one in June 2018 and
one more in June 2019...

> 2019 with 2:2.1.0-6.  Between the two buster was released, and it
> ships
> 2:2.1.0-5 (+deb10u2 right now).  So your upgrade leaps over a debian
> stable release, which is not something supported.  If you had
> upgraded
> from stretch's cryptsetup to buster's *and then* to bullseye/sid's,
> and
> 
> > Also, big upgrades, such as stretch-> buster do not always go
> > cleanly,
> > for which users may resort to performing some degree of selective
> > upgrades in their upgrade path. Would they not potentially run into
> > this error doing `aptitude upgrade cryptsetup` in that situation
> > for
> > such a stretch to buster upgrade in this case?
> 
> No, that's stable → stable+1 so fully supported.
> 
> FYI you don't even need bullseye/sid's cryptsetup for LUKSv2.  It's
> even
> mentioned in buster's release notes:
> https://www.debian.org/releases/buster/amd64/release-notes/ch-whats-new.en.html#cryptsetup-luks2

Sure, but I was just copying the apt lists and cached packages from my
proper up-to-date sid system to a USB pendrive to make use of in the
old live sid environment on this other system I was working on.

> > I think it should be simple policy that when things get moved about
> > between packages, that a breaks is specified for the older versions
> > to
> > prevent such dpkg file-clash errors that could occur in situations
> > such
> > as selective upgrades, and remains permanently in place for a
> > reasonable number of years to avoid any issues for a reasonably
> > wide
> > range of scenarios.
> 
> And that's what we did, except that we count in release cycles not in
> years.  Passed a full release cycle we decided to drop the clutter,
> because going from oldstable to testing (or stable to stable+2), i.e.
> skipping an upgrade, has never been something supported.  This is
> something stated in each release note, see for instance
> 
>     “Direct upgrades from Debian releases older than 9 (stretch) are
> not
>      supported. Please follow the instructions in the Release Notes
> for
>      Debian 9 to upgrade to Debian 9 first.”
>     — 
> https://www.debian.org/releases/buster/amd64/release-notes/ch-upgrading.en.html
> 
> Or 
> 
>     “Please always upgrade to Debian Testing from current stable.
>     Upgrading from oldstable is not supported and might encounter
>     unexpected errors.”
>     — https://wiki.debian.org/DebianTesting
> 
> Also, the package relationship isn't the only thing to care about
> when
> upgrading.  There are also warning for deprecated settings which are
> later removed, compatibility symlinks, etc.  Skipping (Debian)
> releases
> upon upgrades has never been something supported, and we decided to
> simply remove the clutter.  For the same reason we might just close
> this
> as ‘wontfix’.
> 
> > I expected that a simple oversight had been made at the time the
> > cryptsetup reorganisation was done, though your focus on upgrade
> > paths
> > suggests otherwise, that you perhaps do not give as much
> > consideration
> > to supporting sid & testing installations as stable ones?
> 
> Uh?  testing will be the next stable, so we're very careful about
> what
> goes inside.  And same thing for sid.

> Sorry, a sid system deployed
> before stretch was released, with a direct (selective or full)
> upgrade
> to sid isn't a sid system, that's very much a FrankenDebian.

Okay, I can accept that :P.

And I can accept that adjusting my thinking to relate an old sid
install with how it maps to stable releases and do stepped upgrades
could be sensible.

> > edit: I just tried looking up Debian policy regarding sid/unstable
> > and
> > breaks info. I came across 
> > https://www.debian.org/doc/debian-policy/ch-relationships.html, but
> > nothing specific about 'sid'/'unstable' was mentioned, nor in fact
> > 'stable'... nor in https://www.debian.org/doc/debian-policy/
> 
> We have proper package relationship in place to support upgrading
> from
> stretch to buster (or more precisely, from ≥2:1.7.3-4, <2:2.1.0-5 to
> 2:2.1.0-5)
> and others fro buster to bullseye/sid (or more precisely, from
> ≥2:2.1.0-5, <2:2.2.2-3 to 2:2.2.2-3, or whichever version we'll ship
> to
> bullseye). 

Ok. Again, I'm not so concerned about the upgrade path from the
specific old version that was mentioned in the error shown in the
original submission (though I don't seem to have been at all clear
about this there). I however take on board some point you make about
just how old an upgrade that is and that various compatibility hacks
and such could get dropped in such large jumps in package versions
making it a risky thing to do, and the point that such an old sid
install should perhaps no longer really be thought of as being sid.

I submitted the report having remembered a cryptsetup reorganisation
having taken place, thinking that this was surely related to the error
(which it is of course), and having it in mind that the reorganisation
did not occur very long ago, thus expecting that the necessary breaks
info should have been in place. As expanded upon below, the error would
have occurred with any version up to and including 2:2.0.2-1 from March
2018 and so effectively it's a stretch to bullseye upgrade issue as
you've re-titled the report. You maintain that only stable to next-
stable upgrades are really supported and old compatibility data
promptly removed.

Indeed if I had performed a stable->stable->stable upgrade then I would
not have encountered the error. I only question whether limiting
upgrade compatibility so strictly is the best idea. Especially when a
couple of breaks entries is a such a tiny things for a package to carry
and enhances robustness.

Looking a little more closely than I originally did...
 - 2:2.0.3-1 (June 2018) made an initial package split. A breaks change
was certainly introduced at least in 2:2.0.3-3 if not sooner (per
changelog).
 - 2:2.0.3-3 (June 2018) removed 3+ release-old breaks data from a
previous reorganisation.
 - 2:2.1.0-4 (May 2019) mentions moving luksformat from cryptsetup-bin
to cryptsetup-run.
 - 2:2.1.0-5 (June 2019) seems to be the one that made it into buster
 - 2:2.1.0-6 (July 2019) swapped cryptsetup and cryptsetup-run
packages, leaving cryptsetup-run as a dummy package instead of
cryptsetup.

The current package has breaks '<< 2:2.1.0-6' on cryptsetup-run.

Thus if you have 2:2.0.3-1 (where cryptsetup-run came into existence it
seems) or later, then the breaks will kick in and help guide the user
to a correct upgrade path of the collection of packages. Anything older
and since the package has no breaks on older 'cryptsetup' and/or
'cryptsetup-bin' packages, certain upgrade paths can slip through and
end up with a dpkg file-clash. Thus cryptsetup today only robustly
supports upgrades from up to 20 months ago. If you have a live-disc or
otherwise from anything older than that which includes cryptsetup and
try to upgrade in the way I did then it can run into failure as I did.

>From git history it was in 2:2.1.0-6 that breaks entries for
'cryptsetup' and 'cryptsetup-bin' were removed, replaced with just a
breaks on 'cryptsetup-run'. So back in July 2019 cryptsetup could
perform an upgrade without any possible dpkg issues only with versions
up to 13 months old...

Sure, the 2:2.0.3-1 and 2:2.1.0-6 releases saw a new stable distro
release inbetween, but I don't think that this is reason enough to for
packages to be dropping breaks info. It was only 13 months of properly
robust upgrade support in real terms, which might not be anything like
long enough as a general rule of thumb for managing packages.
Especially when the breaks entries are a tiny piece of info for a
control file to hold. Breaks entries for 'cryptsetup' and 'cryptsetp-
bin' could have just sat there for the next 10-20 years without it
being a bother.

I kind of think that where reorganisation occurs, it should be
considered important that breaks data is added and kept for a
significant period of time (a good 2 or 3 releases) to counter any
issues. Why the need to rush into removing these little pieces of
useful info, making release to release+2 upgrades risky without
significant reason (i.e. maintenance burden, significant cruft buildup,
significant apt version decision algo impact, or similar)?

Anyway, I'll leave it up to you. Add the breaks or don't, it's just one
package (well a few technically) and it's not that important to me.
You're perfectly correct that a stable->stable step-by-step upgrade is
the non-risky path that should be recommended if only stable->newstable 
upgrades are guaranteed. Though it's not like adding it would be
burdensome, and in fact would have saved us the time we've spent
debating this if they'd not been removed :P anyway I feel I'm starting
to go in circles and it's of no value to keep writing. :)