AlexWebsites Posted March 30 Share Posted March 30 Not related to IPS and just throwing this out there to see if anyone has suggestions on how to troubleshoot. My server went down last night and I only found my sites were all offline in the morning. Fast forward and I can't get mysql server to restart. I can get it to restart if I add to /etc/my.cnf innodb_force_recovery = 2 Error log gives a couple of warnings but only one error 2024-03-30T21:08:57.164248Z 0 [ERROR] [MY-013183] [InnoDB] Assertion failure: trx0types.h:541:m_rsegs_n < 2 thread 139670608385792 Ideas on troubleshooting? Error log doesn't give me any corrupt database or files.. I was going to start dropping tables but thinking maybe this is a sign I need to upgrade from Centos 7 to AlmaLinux (running cpanle/whm) and move to a new box. Link to comment Share on other sites More sharing options...
Randy Calvert Posted March 30 Share Posted March 30 (edited) It sounds like your mySQL database might have some corruption in it. You might want to restore it from backup. Since you can get it back online with recovery mode 2, you can just export the database and rescue the data. https://mariadb.com/kb/en/innodb-recovery-modes/ I would suggest immediately dumping the database, andreimport it. (You could potentially just import it into a new database with a new username/password as well just to be safe.) Did the server run out of space or was it rebooted without a clean shutdown? I typically only see that sort of thing happen when mySQL loses the ability to read data that happened to be in the memory buffer before it has a chance to write to the disk. Edited March 30 by Randy Calvert AlexWebsites 1 Link to comment Share on other sites More sharing options...
AlexWebsites Posted March 31 Author Share Posted March 31 (edited) I have backups and would do that but my issue is that I’m not sure which database is causing the issue. I’ve got 13 or so and none of them come up in the error log. I think something happened at the data center overnight and then something got corrupted. Edited March 31 by AlexWebsites Link to comment Share on other sites More sharing options...
Randy Calvert Posted March 31 Share Posted March 31 (edited) You might be able to do something from the command line with something like: for i in $(ls /var/lib/mysql/*/*.ibd) do innochecksum -v $i done That’s assuming the path to your raw db files are in that path. Also I would first try to run that command with MySQL offline first. I’m on my phone now so I can’t fix formatting. Edited March 31 by Randy Calvert AlexWebsites and SeNioR- 2 Link to comment Share on other sites More sharing options...
AlexWebsites Posted April 4 Author Share Posted April 4 This ended up being non-recoverable. I ended up moving my sites to a new setup and trying out plesk over cpanel, so far so good. SeNioR- 1 Link to comment Share on other sites More sharing options...
Recommended Posts