[debian-mysql] Bug#853972: mariadb-10.1: upgrading from mysql-5.7 renders old data inaccessible with no option to abort

[Julian Gilbey]

Package: mariadb-server-10.1
Version: 10.1.21-5
Severity: important
Tags: patch


(Only important because no data is actually lost, but it may actually
be a serious problem.)

This is a problem and possible solution for within the stretch release
cycle which I just discovered during my tests.  It doesn't affect
jessie -> stretch upgrading, because mariadb-10.1 can convert
mysql-5.5 databases.


Scenario: I have mysql-server-5.7 installed, and decide to install
mysql-default-server or mariadb-server or mariadb-server-10.1.

What happens: The mariadb-server-10.1 preinst tells me (lines 114
onwards):

Via debconf:

 A file named /var/lib/mysql/debian-*.flag exists on this system.  The
 number indicates a database binary format version that cannot
 automatically be upgraded (or downgraded).
 
 Therefore the previous data directory will be renamed to
 /var/lib/mysql-* and a new data directory will be initialized at
 /var/lib/mysql.
 
 Please manually export/import your data (e.g. with mysqldump) if
 needed.

and then via direct output to the terminal:

 The file /var/lib/mysql/debian-5.7.flag indicates a
 version that cannot automatically be upgraded. Therefore the
 previous data directory will be renamed to /var/lib/mysql-5.7 and
 a new data directory will be initialized at /var/lib/mysql.
 Please manually export/import your data (e.g. with mysqldump) if needed.
 
It then goes ahead, as soon as I've acknowledged the warning, to move
/var/lib/mysql (as advertised) and /etc/mysql/debian.cnf (with no
warning).

BANG!!!

Oh dear.  I now can't run mysqldump, as mariadb-10.1 can't read
mysql-5.7 databases.  And there's no option to abort the installation
either.


The only thing I can do is purge mariadb-server-10.1 and all other
mariadb-* packages, then reinstall mysql-5.7 packages, and try to run
mysqldump at that point.  I might also be a bit stymied (haven't
checked this one) because of the moving of the /etc/mysql/debian.cnf
file (which mariadb didn't mention), but mysql may be OK anyway on a
fresh installation.


Solution (should work for stretch release cycle and probably beyond):

Firstly, the db_critical input should be a boolean question
(defaulting to false) rather than a note (and it should perhaps
mention /etc/mysql/debian.cnf as well, but I'm not sure about that),
to allow the sysadmin to bail out.

If the sysadmin does bail out, then the preinst will fail and the
installation will abort.  Even though the mysql-server packages will
have been removed by this time (because of the package conflicts), it
is much easier for the sysadmin to just (re)install mysql-5.7 than to
have to undo the mariadb-server installation in addition.


In more detail:

The preinst should split between two cases: in lines 86-108, the two
cases of upgrading from mysql > 5.6 and downgrading from a later
version of mariadb are conflated.  So split them into:

  if dpkg --compare-versions "$found_version" '>>' "$this_version"
  then
    downgrade_detected=true
  fi

  if dpkg --compare-versions "$found_version" '>>' "$max_upgradeable_version" \
  && dpkg --compare-versions "$found_version" '<<' "10.0"
  then
    cant_upgrade_mysql=true
  fi

Then there should be two alternative cases for the db_input critical,
which have the option to bail out, but with different appropriate
advice:

Template: mariadb-server-10.1/downgrade_old_data_directory_saved
Type: boolean
Default: false
_Description: Downgrading mariadb and moving old data directory
 A file named /var/lib/mysql/debian-*.flag exists on this system.
 The number indicates a database binary format version that cannot automatically
 be downgraded.
 .
 Therefore the previous data directory will be renamed to /var/lib/mysql-* and
 a new empty data directory will be initialized at /var/lib/mysql.
 .
 Please manually export/import your data (e.g. with mysqldump) if needed.
 .
 You CANNOT export or read your existing mysql data with the mariadb-server
 package you are about to install.  You will need to reinstall your
 previous mysql server package to be able to export it.
 .
 Should I proceed with this downgrade?  If you are unsure, say NO.

Template: mariadb-server-10.1/mysql_old_data_directory_saved
Type: boolean
Default: false
_Description: Migrating from mysql and moving old data directory
 A file named /var/lib/mysql/debian-*.flag exists on this system.
 The number indicates a database binary format version that cannot automatically
 be converted to the mariadb-server format.
 .
 Therefore the previous data directory will be renamed to /var/lib/mysql-* and
 a new empty data directory will be initialized at /var/lib/mysql.
 .
 Please manually export/import your data (e.g. with mysqldump) if needed.
 .
 You CANNOT export or read your existing mysql data with the mariadb-server
 package you are about to install.  You will need to reinstall a
 mysql server package (either your previous one or a newer one) to be able to
 export it.
 .
 Should I proceed with this mariadb installation?  If you are unsure, say NO.


Then the preinst code handles the two cases separately:

if [ ! -z "$downgrade_detected" ]
then
  db_input critical mariadb-server-10.1/downgrade_old_data_directory_saved || true
  db_go
  db_get mariadb-server-10.1/downgrade_old_data_directory_saved
  # Bail out at this point if requested
  if [ "$RET" != "true" ]; then exit 1; fi

  echo "The file $mysql_datadir/debian-$found_version.flag indicates a" 1>&2
  echo "version that cannot automatically be downgraded. Therefore the" 1>&2
  echo "previous data directory will be renamed to $mysql_datadir-$found_version and" 1>&2
  echo "a new data directory will be initialized at $mysql_datadir." 1>&2
  mv -f $mysql_datadir $mysql_datadir-$found_version
  # Also move away the old debian.cnf file that included credentials that are
  # no longer valid
  mv -f /etc/mysql/debian.cnf /etc/mysql/debian.cnf-$found_version
fi

if [ ! -z "$cant_upgrade_mysql" ]
then
  db_input critical mariadb-server-10.1/mysql_old_data_directory_saved || true
  db_go
  db_get mariadb-server-10.1/mysql_old_data_directory_saved
  # Bail out at this point if requested
  if [ "$RET" != "true" ]; then exit 1; fi

  echo "The file $mysql_datadir/debian-$found_version.flag indicates a" 1>&2
  echo "version that cannot automatically be converted. Therefore the" 1>&2
  echo "previous data directory will be renamed to $mysql_datadir-$found_version and" 1>&2
  echo "a new data directory will be initialized at $mysql_datadir." 1>&2
  mv -f $mysql_datadir $mysql_datadir-$found_version
  # Also move away the old debian.cnf file that included credentials that are
  # no longer valid
  mv -f /etc/mysql/debian.cnf /etc/mysql/debian.cnf-$found_version
fi


I think this will even work for stretch -> buster, as I assume that
mysql-5.8 or mysql-5.9 will still be able to read mysql-5.7 data
directories if the sysadmin wants to go back.

Best wishes,

   Julian