[debian-mysql] Bug#862103: mariadb-server-10.1: MariaDB crash after throwing an instance of 'std::out_of_range'

Sam D. cpcbugreport at cpc.cx
Mon May 8 14:32:53 UTC 2017


Package: mariadb-server-10.1
Version: 10.1.22-3
Severity: grave
Tags: newcomer upstream
Justification: causes non-serious data loss

Dear Maintainer,

A critical MySQL bug was discovered in InnoDB storage engine (related to statistics calculation) some weeks ago.

This bug affects MariaDB 10.1 as well and is present in the current Debian Stretch (MariaDB 10.1.22) package 
[mariadb  Ver 15.1 Distrib 10.1.22-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2]. 

Seems to also affect the unstable 10.1.23-3 package. 

MariaDB crash very often on many of our server, from many times daily to many times hourly.

The bug (and fix) is well described here :

https://bugs.mysql.com/bug.php?id=84940 (rated serious)

and here

https://jira.mariadb.org/browse/MDEV-12281 (rated critical)

The bug come from a missing bracket in the InnoDB Stats code and a patch is available here :

https://github.com/mysql/mysql-server/commit/0241e1c1b216d87050cdfcb4531ffc17f9eb1dc7

Data corruption in InnoDB tables may occur as a side effect.


Typical Error/crash log :

-------------------------------------------------------------------------------------------

Version: '10.1.22-MariaDB-'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  Debian 9.0
terminate called after throwing an instance of 'std::out_of_range'
  what():  vector::_M_range_check: __n (which is 4294967295) >= this->size() (which is 0)
170508 13:55:21 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see https://mariadb.com/kb/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.

Server version: 10.1.22-MariaDB-
key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=5
max_threads=386
thread_count=2
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 864227 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x2e)[0x5607ce9d50ce]
/usr/sbin/mysqld(handle_fatal_signal+0x3bd)[0x5607ce51e50d]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x110c0)[0x7feadffc90c0]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcf)[0x7feade8c4fcf]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7feade8c63fa]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x15d)[0x7feadefc50ad]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8f066)[0x7feadefc3066]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8f0b1)[0x7feadefc30b1]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8f2c9)[0x7feadefc32c9]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZSt24__throw_out_of_range_fmtPKcz+0xf5)[0x7feadefebb85]
/usr/sbin/mysqld(+0x9ace97)[0x5607ce931e97]
/usr/sbin/mysqld(+0x9b07f4)[0x5607ce9357f4]
/usr/sbin/mysqld(+0x9b4626)[0x5607ce939626]
/usr/sbin/mysqld(+0x9b5d19)[0x5607ce93ad19]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7494)[0x7feadffbf494]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7feade97a93f]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: The InnoDB memory heap is disabled
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: Compressed tables use zlib 1.2.8
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: Using Linux native AIO
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: Using SSE crc32 instructions
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: Initializing buffer pool, size = 2.0G
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: Completed initialization of buffer pool
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: Highest supported file format is Barracuda.
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: Starting crash recovery from checkpoint LSN=195680453668
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: Restoring possible half-written data pages from the doublewrite buffer...
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 1 row operations to undo
InnoDB: Trx id counter is 100954368
2017-05-08 13:55:27 140012981801536 [Note] InnoDB: Starting final batch to recover 107 pages from redo log
2017-05-08 13:55:28 140012981801536 [Note] InnoDB: 128 rollback segment(s) are active.
InnoDB: Starting in background the rollback of uncommitted transactions
2017-05-08 13:55:28 7f56ab3fe700  InnoDB: Rolling back trx with id 100953855, 1 rows to undo
2017-05-08 13:55:28 140012981801536 [Note] InnoDB: Waiting for purge to start
2017-05-08 13:55:28 140010217006848 [Note] InnoDB: Rollback of trx with id 100953855 completed
2017-05-08 13:55:28 7f56ab3fe700  InnoDB: Rollback of non-prepared transactions completed
2017-05-08 13:55:28 140012981801536 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.35-80.0 started; log sequence number 195733664191
2017-05-08 13:55:28 140012981801536 [Note] Plugin 'FEEDBACK' is disabled.
2017-05-08 13:55:28 140010124715776 [Note] InnoDB: Dumping buffer pool(s) not yet started
2017-05-08 13:55:28 140012981801536 [Note] Recovering after a crash using tc.log
2017-05-08 13:55:28 140012981801536 [Note] Starting crash recovery...
2017-05-08 13:55:28 140012981801536 [Note] Crash recovery finished.
2017-05-08 13:55:28 140012981801536 [Note] Server socket created on IP: '127.0.0.1'.
2017-05-08 13:55:28 140012981801536 [Note] /usr/sbin/mysqld: ready for connections.

-------------------------------------------------------------------------------------

It's my first bug report here, sorry if I miss something.



-- System Information:
Debian Release: 9.0
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64
 (x86_64)

Kernel: Linux 4.9.0-2-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages mariadb-server-10.1 depends on:
ii  adduser                   3.115
ii  debconf [debconf-2.0]     1.5.60
ii  galera-3 [galera-3]       25.3.20-stretch
ii  gawk                      1:4.1.4+dfsg-1
ii  init-system-helpers       1.48
ii  iproute2                  4.9.0-1
ii  libaio1                   0.3.110-3
ii  libc6                     2.24-10
ii  libdbi-perl               1.636-1+b1
ii  libpam0g                  1.1.8-3.5
ii  libstdc++6                6.3.0-16
ii  lsb-base                  9.20161125
ii  lsof                      4.89+dfsg-0.1
ii  mariadb-client-10.1       10.1.22-3
ii  mariadb-common            10.1.23+maria-1~stretch
ii  mariadb-server-core-10.1  10.1.22-3
ii  passwd                    1:4.4-4
ii  perl                      5.24.1-2
ii  psmisc                    22.21-2.1+b2
ii  rsync                     3.1.2-1
ii  socat                     1.7.3.1-2+b1
ii  zlib1g                    1:1.2.8.dfsg-5

Versions of packages mariadb-server-10.1 recommends:
ii  libhtml-template-perl  2.95-2

Versions of packages mariadb-server-10.1 suggests:
ii  bsd-mailx [mailx]  8.1.2-0.20160123cvs-4
pn  mariadb-test       <none>
pn  netcat-openbsd     <none>
pn  tinyca             <none>

-- Configuration Files:
/etc/mysql/mariadb.conf.d/50-server.cnf changed:
[server]
[mysqld]
user		= mysql
pid-file	= /var/run/mysqld/mysqld.pid
socket		= /var/run/mysqld/mysqld.sock
port		= 3306
basedir		= /usr
datadir		= /var/lib/mysql
tmpdir		= /tmp
lc-messages-dir	= /usr/share/mysql
skip-external-locking
bind-address		= 127.0.0.1
key_buffer_size		= 16M
max_allowed_packet	= 16M
thread_stack		= 192K
thread_cache_size       = 8
myisam_recover_options  = BACKUP
max_connections        = 384
query_cache_limit	= 1M
query_cache_size        = 16M
log_error = /var/log/mysql/error.log
slow_query_log = 1
slow_query_log_file	= /var/log/mysql/mariadb-slow.log
long_query_time = 2
log_slow_rate_limit	= 1000
log_slow_verbosity	= query_plan
expire_logs_days	= 10
max_binlog_size   = 100M
innodb_buffer_pool_size = 2048M
character-set-server  = utf8mb4
collation-server      = utf8mb4_general_ci
[embedded]
[mariadb]
[mariadb-10.1]


-- debconf information excluded



More information about the pkg-mysql-maint mailing list