MariaDB startet nach Update nicht mehr

Installiert ist MariaDB Version 10.1.41 auf Debian. Nach einem Update konnte MariaDB nicht mehr gestartet werden:

# systemctl status mariadb.service
● mariadb.service - MariaDB 10.1.41 database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2019-10-08 22:31:44 CEST; 11min ago
     Docs: man:mysqld(8)
           https://mariadb.com/kb/en/library/systemd/
  Process: 21873 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER $_WSREP_START_POSITION (code=exited, status=1/FAILURE)
  Process: 21804 ExecStartPre=/bin/sh -c [ ! -e /usr/bin/galera_recovery ] && VAR= ||   VAR=`/usr/bin/galera_recovery`; [ $? -eq 0 ]   && syst
  Process: 21798 ExecStartPre=/bin/sh -c systemctl unset-environment _WSREP_START_POSITION (code=exited, status=0/SUCCESS)
  Process: 21797 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS)
 Main PID: 21873 (code=exited, status=1/FAILURE)
   Status: "MariaDB server is down"
      CPU: 655ms

Okt 08 22:31:40 webserver systemd[1]: Starting MariaDB 10.1.41 database server...
Okt 08 22:31:41 webserver mysqld[21873]: 2019-10-08 22:31:41 139969245212032 [Note] /usr/sbin/mysqld (mysqld 10.1.41-MariaDB-0+deb9u1) starting
Okt 08 22:31:44 webserver systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Okt 08 22:31:44 webserver systemd[1]: Failed to start MariaDB 10.1.41 database server.
Okt 08 22:31:44 webserver systemd[1]: mariadb.service: Unit entered failed state.
Okt 08 22:31:44 webserver systemd[1]: mariadb.service: Failed with result 'exit-code'.

Nochmal MariaDB / MySQL starten:

# /etc/init.d/mysql start

Journalctl

Da es ja nicht startet kann man den Fehler im Journal nun direkt sehen:

# journalctl -xe
Okt 08 22:49:33 webserver systemd[1]: Starting MariaDB 10.1.41 database server...
-- Subject: Unit mariadb.service has begun start-up
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit mariadb.service has begun starting up.
Okt 08 22:49:33 webserver mysqld[27258]: 2019-10-08 22:49:33 140668169600384 [Note] /usr/sbin/mysqld (mysqld 10.1.41-MariaDB-0+deb9u1) starting
Okt 08 22:49:36 webserver systemd[1]: mariadb.service: Main process exited, code=exited, status=1/FAILURE
Okt 08 22:49:36 webserver systemd[1]: Failed to start MariaDB 10.1.41 database server.
-- Subject: Unit mariadb.service has failed
-- Defined-By: systemd
-- Support: https://www.debian.org/support
--
-- Unit mariadb.service has failed.
--
-- The result is failed.
Okt 08 22:49:36 webserver systemd[1]: mariadb.service: Unit entered failed state.
Okt 08 22:49:36 webserver systemd[1]: mariadb.service: Failed with result 'exit-code'.

Logfile

Wo liegt das error.log:

# mysqld --help --verbose | grep 'log-error' | tail -1
2019-10-08 22:46:13 140688313216384 [Note] Using unique option prefix 'myisam-recover' is error-prone and can break in the future. Please use the full name 'myisam-recover-options' instead.
2019-10-08 22:46:13 140688313216384 [Note] Plugin 'FEEDBACK' is disabled.
log-error                                                  /var/log/mysql/error.log

Was zeigt das log:

# cat /var/log/mysql/error.log
2019-10-08 22:49:33 140668169600384 [Note] Using unique option prefix 'myisam-recover' is error-prone and can break in the future. Please use the full name 'myisam-recover-options' instead.
2019-10-08 22:49:33 140668169600384 [Note] InnoDB: innodb_empty_free_list_algorithm has been changed to legacy because of small buffer pool size. In order to use backoff, increase buffer pool at least up to 20MB.

2019-10-08 22:49:33 140668169600384 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2019-10-08 22:49:33 140668169600384 [Note] InnoDB: The InnoDB memory heap is disabled
2019-10-08 22:49:33 140668169600384 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-10-08 22:49:33 140668169600384 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2019-10-08 22:49:33 140668169600384 [Note] InnoDB: Compressed tables use zlib 1.2.8
2019-10-08 22:49:33 140668169600384 [Note] InnoDB: Using Linux native AIO
2019-10-08 22:49:33 140668169600384 [Note] InnoDB: Using generic crc32 instructions
2019-10-08 22:49:33 140668169600384 [Note] InnoDB: Initializing buffer pool, size = 128.0M
2019-10-08 22:49:33 140668169600384 [Note] InnoDB: Completed initialization of buffer pool
2019-10-08 22:49:33 140668169600384 [Note] InnoDB: Highest supported file format is Barracuda.
2019-10-08 22:49:34 140668169600384 [Note] InnoDB: 128 rollback segment(s) are active.
2019-10-08 22:49:34 140668169600384 [Note] InnoDB: Waiting for purge to start
2019-10-08 22:49:34 140668169600384 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.44-86.0 started; log sequence number 133637874
2019-10-08 22:49:34 140668169600384 [Note] Plugin 'FEEDBACK' is disabled.
2019-10-08 22:49:34 140668169600384 [Note] Recovering after a crash using tc.log
2019-10-08 22:49:34 140668169600384 [ERROR] Can't init tc log
2019-10-08 22:49:34 140668169600384 [ERROR] Aborting

Ursache

Nachdem ich mehrere Stunden nach der Ursache und einer möglichen Lösung gesucht hatte, kam ich auf die Meldung:

Recovering after a crash using tc.log
2019-10-08 22:49:34 140668169600384 [ERROR] Can't init tc log
2019-10-08 22:49:34 140668169600384 [ERROR] Aborting

Lösung

Die Lösung des gesamten Problems war dann:

# find / -name tc.log
/var/lib/mysql/tc.log

root@webserver:~# rm /var/lib/mysql/tc.log
root@webserver:~# systemctl start mariadb
root@webserver:~# systemctl status mariadb
● mariadb.service - MariaDB 10.1.41 database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2019-10-08 23:51:57 CEST; 5s ago
     Docs: man:mysqld(8)

Danach kann das Update von MariaDB installiert werden:

# apt update && apt upgrade

Thats it – have fun.