[debian-mysql] Bug#914172: Bug#914172: Bug#914172: mariadb-server-10.1: mariadb-server sec-update (10.1.37-0+deb9u1) uninstalls default-mysql-server, mysql-server, mariadb-server-10.1 & mariadb-client-10.1

[Jeremy Davis]

Hi Olaf and Faustin,

@Olaf

Thanks for your quick response and suggestions.

On 20/11/18 18:50, Olaf van der Spek wrote:
> IMO apt shouldn't be run in such a way that packages get removed
> automatically though..

If you have any specific suggestions on how to ensure that apt won't
remove packages, I'd be interested to hear.

Also, IIRC there have been cases where removal of old packages were
required (I think that was the case with Samba security updates within
Jessie?! - Although perhaps I am confused).

@Faustin

Thanks to you too for your prompt reply, apologies that my response has
been a little slow...

I'll aim to provide as much relevant info as possible, if there is
anything else you need please ask. Hopefully it's not too waffley and/or
includes too much irrelevant info... (I'm often told that I need to turn
verbosity down...)

On 21/11/18 04:57, Faustin Lammler wrote:
> Are you able to provide a step-by-step procedure?
> 

Possibly the easiest way to reproduce the issue would be to download our
v15.0 (Stretch based) LAMP appliance ISO[1] (signed hash file here[2])
and install it to a VM and NOT run the initial firstboot "security
updates" script (i.e. select "skip" when asked). Once logged in as root,
you can then poke around inside and see exactly what is going on.

[1]
http://mirror.turnkeylinux.org/turnkeylinux/images/iso/turnkey-lamp-15.0-stretch-amd64.iso
[2]
http://mirror.turnkeylinux.org/turnkeylinux/images/iso/turnkey-lamp-15.0-stretch-amd64.iso.hash

Then the issue can be reproduced by running
'turnkey-install-security-updates' That will only install updates from
Debian (and TurnKey) security repos.

----

In an effort to assist you to avoid that though, here's some more info
which may help.

The process that turnkey-install-security-updates uses is a little
convoluted, but essentially it runs this:

apt-get update
apt-get autoclean -y
apt-get dist-upgrade -y -o APT::Get::Show-Upgraded=true \
 -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/security.sources.list \
 -o Dir::Etc::sourceparts=nonexistent \
 -o DPkg::Options::=--force-confdef \
 -o DPkg::Options::=--force-confold

FWIW the security.sources.list:

deb http://archive.turnkeylinux.org/debian stretch-security main

deb http://security.debian.org/ stretch/updates main
deb http://security.debian.org/ stretch/updates contrib
#deb http://security.debian.org/ stretch/updates non-free

If I run the above dist-upgrade command (after apt-get update and
without the -y switch), I get this:

Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer
required:
  galera-3 libaio1 libjemalloc1 lsof mariadb-client-core-10.1
mariadb-common mariadb-server-core-10.1 socat
Use 'apt autoremove' to remove them.
The following packages will be REMOVED:
  default-mysql-server mariadb-client-10.1 mariadb-server-10.1 mysql-server
The following NEW packages will be installed:
  linux-image-4.9.0-8-amd64
The following packages will be upgraded:
  curl git git-core git-man libcurl3 libcurl3-gnutls libfuse2
libmariadbclient18 libpython2.7 libpython2.7-minimal libpython2.7-stdlib
libpython3.5-minimal libpython3.5-stdlib
  linux-image-4.9.0-7-amd64 linux-image-amd64 mariadb-client-core-10.1
mariadb-common mariadb-server-core-10.1 openssh-client openssh-server
openssh-sftp-server python2.7 python2.7-minimal python3.5
  python3.5-minimal ssh
26 upgraded, 1 newly installed, 4 to remove and 0 not upgraded.
Need to get 107 MB of archives.
After this operation, 68.6 MB of additional disk space will be used.
Do you want to continue? [Y/n]

Obviously some of those packages are irrelevant to this issue, but
figured it best to not omit anything.

If I then allow it to install those updates (and uninstall
default-mysql-server mariadb-client-10.1 mariadb-server-10.1 &
mysql-server), then reinstall default-mysql-server, here's what I get:

root at lamp ~# apt-get install default-mysql-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
  libconfig-inifiles-perl mariadb-client-10.1 mariadb-server-10.1
Suggested packages:
  mariadb-test netcat-openbsd tinyca
Recommended packages:
  libterm-readkey-perl libhtml-template-perl
The following NEW packages will be installed:
  default-mysql-server libconfig-inifiles-perl mariadb-client-10.1
mariadb-server-10.1
0 upgraded, 4 newly installed, 0 to remove and 25 not upgraded.
Need to get 11.3 MB of archives.
After this operation, 125 MB of additional disk space will be used.

So it appears likely that the offending dependency (as suggested by
Olaf) is libconfig-inifiles-perl ?!

> If not, dpgk -l could help to understand what apt dependencies may be
> problematic. And what happened when you reinstalled mariadb. Which
> command did you used, what was the output?
> 

Alternatively (or as well as?), you can see all the packages installed
by apt/dpkg in the appliance manifest[3].

[3]
http://mirror.turnkeylinux.org/turnkeylinux/metadata/turnkey-lamp/15.0-stretch-amd64/turnkey-lamp-15.0-stretch-amd64.manifest

If you would prefer a full dpkg -l output, I'm happy to also provide that.

As noted above, it appears that offending dependency is
libconfig-inifiles-perl. It is not installed (at all) by default on our
servers. Prior to (re)installing default-mysql-server:

root at lamp ~# apt policy libconfig-inifiles-perl
libconfig-inifiles-perl:
  Installed: (none)
  Candidate: 2.94-1
  Version table:
     2.94-1 500
        500 http://deb.debian.org/debian stretch/main amd64 Packages

Below, I've noted more details of upgradable packages from the above
LAMP appliance prior to the upgrade. As above, obviously some which are
irrelevant.

apache2/stable 2.4.25-3+deb9u6 amd64 [upgradable from: 2.4.25-3+deb9u5]
apache2-bin/stable 2.4.25-3+deb9u6 amd64 [upgradable from: 2.4.25-3+deb9u5]
apache2-data/stable 2.4.25-3+deb9u6 all [upgradable from: 2.4.25-3+deb9u5]
apache2-utils/stable 2.4.25-3+deb9u6 amd64 [upgradable from:
2.4.25-3+deb9u5]
base-files/stable 9.9+deb9u6 amd64 [upgradable from: 9.9+deb9u5]
confconsole/stretch 1.1.0+2+g6c2aad9 all [upgradable from: 1.1.0]
curl/stable,stable 7.52.1-5+deb9u8 amd64 [upgradable from: 7.52.1-5+deb9u6]
git/stable,stable 1:2.11.0-3+deb9u4 amd64 [upgradable from:
1:2.11.0-3+deb9u3]
git-core/stable,stable 1:2.11.0-3+deb9u4 all [upgradable from:
1:2.11.0-3+deb9u3]
git-man/stable,stable 1:2.11.0-3+deb9u4 all [upgradable from:
1:2.11.0-3+deb9u3]
gnupg/stable 2.1.18-8~deb9u3 amd64 [upgradable from: 2.1.18-8~deb9u2]
gnupg-agent/stable 2.1.18-8~deb9u3 amd64 [upgradable from: 2.1.18-8~deb9u2]
gpgv/stable 2.1.18-8~deb9u3 amd64 [upgradable from: 2.1.18-8~deb9u2]
grub-common/stable 2.02~beta3-5+deb9u1 amd64 [upgradable from: 2.02~beta3-5]
grub-pc/stable 2.02~beta3-5+deb9u1 amd64 [upgradable from: 2.02~beta3-5]
grub-pc-bin/stable 2.02~beta3-5+deb9u1 amd64 [upgradable from: 2.02~beta3-5]
grub2-common/stable 2.02~beta3-5+deb9u1 amd64 [upgradable from:
2.02~beta3-5]
hdparm/stable 9.51+ds-1+deb9u1 amd64 [upgradable from: 9.51+ds-1]
libcurl3/stable,stable 7.52.1-5+deb9u8 amd64 [upgradable from:
7.52.1-5+deb9u6]
libcurl3-gnutls/stable,stable 7.52.1-5+deb9u8 amd64 [upgradable from:
7.52.1-5+deb9u6]
libfuse2/stable 2.9.7-1+deb9u2 amd64 [upgradable from: 2.9.7-1]
libgnutls30/stable 3.5.8-5+deb9u4 amd64 [upgradable from: 3.5.8-5+deb9u3]
libmariadbclient18/stable 10.1.37-0+deb9u1 amd64 [upgradable from:
10.1.26-0+deb9u1]
libpython2.7/stable,stable 2.7.13-2+deb9u3 amd64 [upgradable from:
2.7.13-2+deb9u2]
libpython2.7-minimal/stable,stable 2.7.13-2+deb9u3 amd64 [upgradable
from: 2.7.13-2+deb9u2]
libpython2.7-stdlib/stable,stable 2.7.13-2+deb9u3 amd64 [upgradable
from: 2.7.13-2+deb9u2]
libpython3.5-minimal/stable,stable 3.5.3-1+deb9u1 amd64 [upgradable
from: 3.5.3-1]
libpython3.5-stdlib/stable,stable 3.5.3-1+deb9u1 amd64 [upgradable from:
3.5.3-1]
libseccomp2/stable 2.3.1-2.1+deb9u1 amd64 [upgradable from: 2.3.1-2.1]
libsystemd0/stable 232-25+deb9u6 amd64 [upgradable from: 232-25+deb9u4]
libudev1/stable 232-25+deb9u6 amd64 [upgradable from: 232-25+deb9u4]
linux-image-4.9.0-7-amd64/stable 4.9.110-3+deb9u2 amd64 [upgradable
from: 4.9.110-1]
linux-image-amd64/stable,stable 4.9+80+deb9u6 amd64 [upgradable from:
4.9+80+deb9u5]
mariadb-client-10.1/stable 10.1.37-0+deb9u1 amd64 [upgradable from:
10.1.26-0+deb9u1]
mariadb-client-core-10.1/stable 10.1.37-0+deb9u1 amd64 [upgradable from:
10.1.26-0+deb9u1]
mariadb-common/stable 10.1.37-0+deb9u1 all [upgradable from:
10.1.26-0+deb9u1]
mariadb-server-10.1/stable 10.1.37-0+deb9u1 amd64 [upgradable from:
10.1.26-0+deb9u1]
mariadb-server-core-10.1/stable 10.1.37-0+deb9u1 amd64 [upgradable from:
10.1.26-0+deb9u1]
openssh-client/stable,stable 1:7.4p1-10+deb9u4 amd64 [upgradable from:
1:7.4p1-10+deb9u3]
openssh-server/stable,stable 1:7.4p1-10+deb9u4 amd64 [upgradable from:
1:7.4p1-10+deb9u3]
openssh-sftp-server/stable,stable 1:7.4p1-10+deb9u4 amd64 [upgradable
from: 1:7.4p1-10+deb9u3]
python2.7/stable,stable 2.7.13-2+deb9u3 amd64 [upgradable from:
2.7.13-2+deb9u2]
python2.7-minimal/stable,stable 2.7.13-2+deb9u3 amd64 [upgradable from:
2.7.13-2+deb9u2]
python3.5/stable,stable 3.5.3-1+deb9u1 amd64 [upgradable from: 3.5.3-1]
python3.5-minimal/stable,stable 3.5.3-1+deb9u1 amd64 [upgradable from:
3.5.3-1]
ssh/stable,stable 1:7.4p1-10+deb9u4 all [upgradable from: 1:7.4p1-10+deb9u3]
systemd/stable 232-25+deb9u6 amd64 [upgradable from: 232-25+deb9u4]
systemd-sysv/stable 232-25+deb9u6 amd64 [upgradable from: 232-25+deb9u4]
tklbam/stretch 1.4.1+37+g8117cd6 all [upgradable from: 1.4.1+32+g07acc1c]
tzdata/stable 2018g-0+deb9u1 all [upgradable from: 2018e-0+deb9u1]
udev/stable 232-25+deb9u6 amd64 [upgradable from: 232-25+deb9u4]

> Olaf's guess maybe true but there can be lot's of reason why APT decided
> to remove mariadb-* packages.
> 

As noted above, it appears that Olaf was right. The offending dependency
appears to be libconfig-inifiles-perl.

FWIW libconfig-inifiles-perl appears to be a (new?) dependency of
mariadb-client-10.1, which is in turn a dependency of mariadb-server-10.1:

root at lamp ~# apt-cache depends mariadb-client-10.1 | grep
libconfig-inifiles-perl
  Depends: libconfig-inifiles-perl
root at lamp ~# apt-cache depends mariadb-server-10.1 | grep
mariadb-client-10.1
  Depends: mariadb-client-10.1

My guess is that this dependency was previously a "recommends" and is
now a hard "depends" (see below). Does that seem likely?

> Finally, do you at turnkeylinux set some non default apt preferences?

Assuming you mean config in general (rather than strictly "preferences")
yes we do. I suspect the one that may have been a causal fact in this
case is not installing "recommends" by default:

root at lamp ~# cat /etc/apt/apt.conf.d/05recommends
// Don't consider recommends as dependencies and install them by default
APT::Install-Recommends "false";

There are some other conf snippets in apt.conf.d, but without further
checking (i.e. comparison with Debian default) I'm not 100% sure which
ones are default and which are added by us. Even then, I'm not sure
which ones are significant? On face value, none seem to be of real
significance in this scenario. E.g.:

root at lamp ~# cat /etc/apt/apt.conf.d/70debconf
// Pre-configure all packages with debconf before they are installed.
// If you don't like it, comment it out.
DPkg::Pre-Install-Pkgs {"/usr/sbin/dpkg-preconfigure --apt || true";};

Having said that I note that there is a 01autoremove script which notes
some packages to never autoremove. Might be worth further investiagation
to mitigate against the cahnce of this in the future?

Hope that is all useful and relevant. Anything further you need, please ask.

Regards,
Jeremy

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/pkg-mysql-maint/attachments/20181121/f1d5f115/attachment-0001.sig>