[debian-mysql] Bug#1009290: Bug#1009290: mariadb-server-10.6: Fails to start on live system
Daniel Lewart
lewart3 at gmail.com
Mon Jan 30 08:08:00 GMT 2023
Otto, et al,
> I don't see anything new in upstream
> https://jira.mariadb.org/browse/MDEV-28751 about this.
> However we do have MariaDB 10.11.1 in Debian now. Maybe you Daniel can
> for the sake of it check if the behaviour is still same on 10.11?
Unfortunately, still the same.
I created a live standard image with CODENAME=unstable:
$ uname -a
Linux debian 6.1.0-2-amd64 #1 SMP PREEMPT_DYNAMIC
Debian 6.1.7-1 (2023-01-18) x86_64 GNU/Linux
Then I did the following:
$ sudo apt -y install mariadb-server
Output included:
Setting up mariadb-server (1:10.11.1-1) ...
Created symlink
/etc/systemd/system/multi-user.target.wants/mariadb.service →
/lib/systemd/system/mariadb.service.
Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 145.
Below is the output from the systemd journal.
In the upstream bug:
Daniel Black added a comment - 2022-06-07 04:50
I'll be just pushing this to the kernel (overlayfs) developers.
I don't know what happened with that.
The good news is that Arnaud R's upstream "innodb_flush_method = fsync"
workaround still works.
Sorry!
Daniel Lewart
Urbana, Illinois
---
$ sudo journalctl -t mariadb-server.postinst | cut -c55-
2023-01-30 0:54:27 0 [Warning] InnoDB: Retry attempts for writing
partial data failed.
2023-01-30 0:54:27 0 [ERROR] InnoDB: Write to file ./ibdata1 failed
at offset 0, 1048576 bytes should have been written, only 0 were
written. Operating system error number 22. Check that your OS and file
system support files of this size. Check also that the disk is not
full or a disk quota exceeded.
2023-01-30 0:54:27 0 [ERROR] InnoDB: Error number 22 means 'Invalid argument'
2023-01-30 0:54:27 0 [ERROR] InnoDB: Could not set the file size of
'./ibdata1'. Probably out of disk space
2023-01-30 0:54:27 0 [ERROR] InnoDB: Database creation was aborted
with error Generic error. You may need to delete the ibdata1 file
before trying to start up again.
2023-01-30 0:54:28 0 [ERROR] InnoDB: Operating system error number 22
in a file operation.
2023-01-30 0:54:28 0 [ERROR] InnoDB: Error number 22 means 'Invalid argument'
2023-01-30 0:54:28 0 [ERROR] InnoDB: File (unknown): 'close' returned
OS error 222. Cannot continue operation
230130 0:54:28 [ERROR] mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
To report this bug, see https://mariadb.com/kb/en/reporting-bugs
We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed,
something is definitely wrong and this may fail.
Server version: 10.11.1-MariaDB-1
key_buffer_size=134217728
read_buffer_size=131072
max_used_connections=0
max_threads=153
thread_count=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads =
468018 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x49000
2023-01-30 0:54:28 0 [ERROR] InnoDB: Operating system error number 9
in a file operation.
2023-01-30 0:54:28 0 [ERROR] InnoDB: Error number 9 means 'Bad file descriptor'
2023-01-30 0:54:28 0 [ERROR] InnoDB: File (unknown): 'close' returned
OS error 209. Cannot continue operation
Printing to addr2line failed
/usr/sbin/mariadbd(my_print_stacktrace+0x2e)[0x55ba61bd403e]
/usr/sbin/mariadbd(handle_fatal_signal+0x409)[0x55ba61741c69]
2023-01-30 0:54:28 0 [ERROR] InnoDB: Operating system error number 9
in a file operation.
2023-01-30 0:54:28 0 [ERROR] InnoDB: Error number 9 means 'Bad file descriptor'
2023-01-30 0:54:28 0 [ERROR] InnoDB: File (unknown): 'close' returned
OS error 209. Cannot continue operation
/lib/x86_64-linux-gnu/libc.so.6(+0x3bf90)[0x7f028b392f90]
/lib/x86_64-linux-gnu/libc.so.6(+0x8accc)[0x7f028b3e1ccc]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x12)[0x7f028b392ef2]
/lib/x86_64-linux-gnu/libc.so.6(abort+0xd3)[0x7f028b37d472]
2023-01-30 0:54:28 0 [ERROR] InnoDB: Operating system error number 9
in a file operation.
2023-01-30 0:54:28 0 [ERROR] InnoDB: Error number 9 means 'Bad file descriptor'
2023-01-30 0:54:28 0 [ERROR] InnoDB: File (unknown): 'close' returned
OS error 209. Cannot continue operation
/usr/sbin/mariadbd(+0x6692e1)[0x55ba613972e1]
/usr/sbin/mariadbd(+0xd0a775)[0x55ba61a38775]
/usr/sbin/mariadbd(+0xe140c8)[0x55ba61b420c8]
/usr/sbin/mariadbd(+0xe14131)[0x55ba61b42131]
/usr/sbin/mariadbd(+0xe16060)[0x55ba61b44060]
/usr/sbin/mariadbd(+0xe17041)[0x55ba61b45041]
/usr/sbin/mariadbd(+0xd7bbae)[0x55ba61aa9bae]
/usr/sbin/mariadbd(+0xcb7545)[0x55ba619e5545]
/usr/sbin/mariadbd(_Z24ha_initialize_handlertonP13st_plugin_int+0x7e)[0x55ba61744e6e]
/usr/sbin/mariadbd(+0x802167)[0x55ba61530167]
/usr/sbin/mariadbd(_Z11plugin_initPiPPci+0x85a)[0x55ba6153112a]
/usr/sbin/mariadbd(+0x6f42e0)[0x55ba614222e0]
/usr/sbin/mariadbd(_Z11mysqld_mainiPPc+0x41d)[0x55ba61427e7d]
2023-01-30 0:54:28 0 [ERROR] InnoDB: Operating system error number 9
in a file operation.
2023-01-30 0:54:28 0 [ERROR] InnoDB: Error number 9 means 'Bad file descriptor'
2023-01-30 0:54:28 0 [ERROR] InnoDB: File (unknown): 'close' returned
OS error 209. Cannot continue operation
/lib/x86_64-linux-gnu/libc.so.6(+0x2718a)[0x7f028b37e18a]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85)[0x7f028b37e245]
2023-01-30 0:54:28 0 [ERROR] InnoDB: Operating system error number 9
in a file operation.
2023-01-30 0:54:28 0 [ERROR] InnoDB: Error number 9 means 'Bad file descriptor'
2023-01-30 0:54:28 0 [ERROR] InnoDB: File (unknown): 'close' returned
OS error 209. Cannot continue operation
/usr/sbin/mariadbd(_start+0x21)[0x55ba6141ceb1]
The manual page at
https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/
contains
information that should help you find out what is causing the crash.
Writing a core file...
Working directory at /var/lib/mysql
Resource Limits:
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 0 bytes
Max resident set unlimited unlimited bytes
Max processes 7693 7693 processes
Max open files 32184 32184 files
Max locked memory 258789376 258789376 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 7693 7693 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Core pattern: core
Kernel version: Linux version 6.1.0-2-amd64
(debian-kernel at lists.debian.org) (gcc-12 (Debian 12.2.0-14) 12.2.0,
GNU ld (GNU Binutils for Debian) 2.40) #1 SMP PREEMPT_DYNAMIC Debian
6.1.7-1 (2023-01-18)
/usr/bin/mysql_install_db: line 554: 2096 Aborted
"$mysqld_bootstrap" $defaults $defaults_group_suffix "$mysqld_opt"
--bootstrap $silent_startup "--basedir=$basedir" "--datadir=$ldata"
--log-warnings=0 --enforce-storage-engine=""
"--plugin-dir=${plugindir}" $args --max_allowed_packet=8M
--net_buffer_length=16K
Installation of system tables failed! Examine the logs in
/var/lib/mysql for more information.
The problem could be conflicting information in an external
my.cnf files. You can ignore these by doing:
shell> /usr/bin/mysql_install_db --defaults-file=~/.my.cnf
You can also try to start the mysqld daemon with:
shell> /usr/sbin/mariadbd --skip-grant-tables --general-log &
and use the command line tool /usr/bin/mariadb
to connect to the mysql database and look at the grant tables:
shell> /usr/bin/mysql -u root mysql
mysql> show tables;
Try 'mysqld --help' if you have problems with paths. Using
--general-log gives you a log in /var/lib/mysql that may be helpful.
The latest information about mysql_install_db is available at
https://mariadb.com/kb/en/installing-system-tables-mysql_install_db
You can find the latest source at https://downloads.mariadb.org and
the maria-discuss email list at https://launchpad.net/~maria-discuss
Please check all of the above before submitting a bug report
at https://mariadb.org/jira
###
More information about the pkg-mysql-maint
mailing list