Invision Community 4: SEO, prepare for v5 and dormant account notifications By Matt Monday at 02:04 PM
virap Posted December 15, 2020 Posted December 15, 2020 Hello, ever since converting to ipb from vbulletin our forum is suffering very slow speeds. Here is the output from mysqltuner Quote >> MySQLTuner 1.7.20 - Major Hayden <major@mhtx.net> >> Bug reports, feature requests, and downloads at http://mysqltuner.pl/ >> Run with '--help' for additional options and output filtering [--] Skipped version check for MySQLTuner script [OK] Currently running supported MySQL version 5.7.29-log [OK] Operating on 64-bit architecture -------- Log file Recommendations ------------------------------------------------------------------ [OK] Log file /var/log/mysqld.log exists [--] Log file: /var/log/mysqld.log(21M) [OK] Log file /var/log/mysqld.log is readable. [OK] Log file /var/log/mysqld.log is not empty [OK] Log file /var/log/mysqld.log is smaller than 32 Mb [!!] /var/log/mysqld.log contains 909 warning(s). [!!] /var/log/mysqld.log contains 122459 error(s). [--] 81 start(s) detected in /var/log/mysqld.log [--] 1) 2020-12-15T01:09:11.047973Z 0 [Note] /usr/sbin/mysqld: ready for connections. [--] 2) 2020-12-14T18:59:39.617572Z 0 [Note] /usr/sbin/mysqld: ready for connections. [--] 3) 2020-12-14T17:32:53.935910Z 0 [Note] /usr/sbin/mysqld: ready for connections. [--] 4) 2020-12-14T15:26:53.151077Z 0 [Note] /usr/sbin/mysqld: ready for connections. [--] 5) 2020-12-14T14:53:06.789355Z 0 [Note] /usr/sbin/mysqld: ready for connections. [--] 6) 2020-12-14T14:49:08.764884Z 0 [Note] /usr/sbin/mysqld: ready for connections. [--] 7) 2020-12-14T14:47:27.643538Z 0 [Note] /usr/sbin/mysqld: ready for connections. [--] 😎 2020-12-14T14:13:38.642713Z 0 [Note] /usr/sbin/mysqld: ready for connections. [--] 9) 2020-12-14T14:04:48.436808Z 0 [Note] /usr/sbin/mysqld: ready for connections. [--] 10) 2020-12-14T13:22:42.675042Z 0 [Note] /usr/sbin/mysqld: ready for connections. [--] 80 shutdown(s) detected in /var/log/mysqld.log [--] 1) 2020-12-15T01:09:09.013032Z 0 [Note] /usr/sbin/mysqld: Shutdown complete [--] 2) 2020-12-14T18:59:37.486039Z 0 [Note] /usr/sbin/mysqld: Shutdown complete [--] 3) 2020-12-14T17:32:51.838492Z 0 [Note] /usr/sbin/mysqld: Shutdown complete [--] 4) 2020-12-14T15:26:51.103689Z 0 [Note] /usr/sbin/mysqld: Shutdown complete [--] 5) 2020-12-14T14:53:04.803733Z 0 [Note] /usr/sbin/mysqld: Shutdown complete [--] 6) 2020-12-14T14:49:06.744722Z 0 [Note] /usr/sbin/mysqld: Shutdown complete [--] 7) 2020-12-14T14:47:25.629317Z 0 [Note] /usr/sbin/mysqld: Shutdown complete [--] 😎 2020-12-14T14:13:36.727847Z 0 [Note] /usr/sbin/mysqld: Shutdown complete [--] 9) 2020-12-14T14:04:46.435991Z 0 [Note] /usr/sbin/mysqld: Shutdown complete [--] 10) 2020-12-14T13:22:40.677051Z 0 [Note] /usr/sbin/mysqld: Shutdown complete -------- Storage Engine Statistics ----------------------------------------------------------------- [--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MEMORY +MRG_MYISAM +MyISAM +PERFORMANCE_SCHEMA [--] Data in MyISAM tables: 10.2G (Tables: 596) [--] Data in InnoDB tables: 9.8G (Tables: 108) [--] Data in MEMORY tables: 0B (Tables: 2) [!!] Total fragmented tables: 3 -------- Analysis Performance Metrics -------------------------------------------------------------- [--] innodb_stats_on_metadata: OFF [OK] No stat updates during querying INFORMATION_SCHEMA. -------- CVE Security Recommendations -------------------------------------------------------------- [OK] NO SECURITY CVE FOUND FOR YOUR VERSION -------- Performance Metrics ----------------------------------------------------------------------- [--] Up for: 20h 41m 1s (10M q [134.742 qps], 367K conn, TX: 61G, RX: 19G) [--] Reads / Writes: 47% / 53% [--] Binary logging is disabled [--] Physical Memory : 94.3G [--] Max MySQL memory : 212.7G [--] Other process memory: 0B [--] Total buffers: 10.4G global + 259.0M per thread (800 max threads) [--] P_S Max memory usage: 72B [--] Galera GCache Max memory usage: 0B [OK] Maximum reached memory usage: 46.5G (49.32% of installed RAM) [!!] Maximum possible memory usage: 212.7G (225.47% of installed RAM) [!!] Overall possible memory usage with other process exceeded memory [OK] Slow queries: 0% (1K/10M) [OK] Highest usage of available connections: 17% (143/800) [OK] Aborted connections: 0.00% (13/367070) [!!] name resolution is active : a reverse name resolution is made for each new connection and can reduce performance [!!] Query cache may be disabled by default due to mutex contention. [OK] Query cache efficiency: 50.7% (3M cached / 5M selects) [!!] Query cache prunes per day: 1302707 [!!] Sorts requiring temporary tables: 837% (1M temp sorts / 236K sorts) [!!] Joins performed without indexes: 4298 [!!] Temporary tables created on disk: 30% (44K on disk / 149K total) [OK] Thread cache hit rate: 99% (143 created / 367K connections) [OK] Table cache hit rate: 55% (8K open / 14K opened) [OK] table_definition_cache(8000) is upper than number of tables(986) [OK] Open file limit used: 35% (7K/20K) [OK] Table locks acquired immediately: 99% (5M immediate / 5M locks) -------- Performance schema ------------------------------------------------------------------------ [--] Memory used by P_S: 72B [--] Sys schema is installed. -------- ThreadPool Metrics ------------------------------------------------------------------------ [--] ThreadPool stat is disabled. -------- MyISAM Metrics ---------------------------------------------------------------------------- [OK] Key buffer used: 100.0% (167M used / 167M cache) [OK] Key buffer size / total MyISAM indexes: 160.0M/2.4G [OK] Read Key buffer hit rate: 100.0% (10B cached / 339K reads) [OK] Write Key buffer hit rate: 97.4% (1M cached / 1M writes) -------- InnoDB Metrics ---------------------------------------------------------------------------- [--] InnoDB is enabled. [--] InnoDB Thread Concurrency: 16 [OK] InnoDB File per table is activated [OK] InnoDB buffer pool / data size: 10.0G/9.8G [!!] Ratio InnoDB log file size / InnoDB Buffer pool size (17.578125 %): 900.0M * 2/10.0G should be equal to 25% [OK] InnoDB buffer pool instances: 10 [--] Number of InnoDB Buffer Pool Chunk : 80 for 10 Buffer Pool Instance(s) [OK] Innodb_buffer_pool_size aligned with Innodb_buffer_pool_chunk_size & Innodb_buffer_pool_instances [OK] InnoDB Read buffer efficiency: 99.99% (4417035141 hits/ 4417317247 total) [OK] InnoDB Write log efficiency: 99.38% (35041392 hits/ 35259208 total) [OK] InnoDB log waits: 0.00% (0 waits / 217816 writes) -------- AriaDB Metrics ---------------------------------------------------------------------------- [--] AriaDB is disabled. -------- TokuDB Metrics ---------------------------------------------------------------------------- [--] TokuDB is disabled. -------- XtraDB Metrics ---------------------------------------------------------------------------- [--] XtraDB is disabled. -------- Galera Metrics ---------------------------------------------------------------------------- [--] Galera is disabled. -------- Replication Metrics ----------------------------------------------------------------------- [--] Galera Synchronous replication: NO [--] No replication slave(s) for this server. [--] Binlog format: ROW [--] XA support enabled: ON [--] Semi synchronous replication Master: Not Activated [--] Semi synchronous replication Slave: Not Activated [--] This is a standalone server -------- Recommendations --------------------------------------------------------------------------- General recommendations: Control warning line(s) into /var/log/mysqld.log file Control error line(s) into /var/log/mysqld.log file Run OPTIMIZE TABLE to defragment tables for better performance OPTIMIZE TABLE `enotal1_enotal1`.`ipb_core_edit_history`; -- can free 524.075992584229 MB OPTIMIZE TABLE `enotal1_enotal1`.`ipb_core_output_cache`; -- can free 159 MB OPTIMIZE TABLE `enotal1_enotal1`.`vb_searchlog`; -- can free 305.898475646973 MB Total freed space after theses OPTIMIZE TABLE : 988.974468231201 Mb Set up a Secure Password for enotal1_pchat1@173.192.125.130 user: SET PASSWORD FOR 'enotal1_pchat1'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat1@192.168.0.200 user: SET PASSWORD FOR 'enotal1_pchat1'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat1@68.173.171.130 user: SET PASSWORD FOR 'enotal1_pchat1'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat1@69.46.2.58 user: SET PASSWORD FOR 'enotal1_pchat1'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat1@cpe-68-173-171-130.nyc.res.rr.com user: SET PASSWORD FOR 'enotal1_pchat1'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat1@localhost user: SET PASSWORD FOR 'enotal1_pchat1'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat1@server.enotalone.com user: SET PASSWORD FOR 'enotal1_pchat1'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat1@server1.enotalone1.com user: SET PASSWORD FOR 'enotal1_pchat1'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat2@173.192.125.130 user: SET PASSWORD FOR 'enotal1_pchat2'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat2@192.168.0.200 user: SET PASSWORD FOR 'enotal1_pchat2'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat2@68.173.171.130 user: SET PASSWORD FOR 'enotal1_pchat2'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat2@69.46.2.58 user: SET PASSWORD FOR 'enotal1_pchat2'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat2@cpe-68-173-171-130.nyc.res.rr.com user: SET PASSWORD FOR 'enotal1_pchat2'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat2@localhost user: SET PASSWORD FOR 'enotal1_pchat2'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat2@server.enotalone.com user: SET PASSWORD FOR 'enotal1_pchat2'@'SpecificDNSorIp' = PASSWORD('secure_password'); Set up a Secure Password for enotal1_pchat2@server1.enotalone1.com user: SET PASSWORD FOR 'enotal1_pchat2'@'SpecificDNSorIp' = PASSWORD('secure_password'); MySQL was started within the last 24 hours - recommendations may be inaccurate Reduce your overall MySQL memory footprint for system stability Dedicate this server to your database for highest performance. Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1 Increasing the query_cache size over 128M may reduce performance We will suggest raising the 'join_buffer_size' until JOINs not using indexes are found. See https://dev.mysql.com/doc/internals/en/join-buffer-size.html (specially the conclusions at the bottom of the page). When making adjustments, make tmp_table_size/max_heap_table_size equal Reduce your SELECT DISTINCT queries which have no LIMIT clause Before changing innodb_log_file_size and/or innodb_log_files_in_group read this: https://bit.ly/2TcGgtU Variables to adjust: *** MySQL's maximum memory usage is dangerously high *** *** Add RAM before increasing MySQL buffer variables *** query_cache_size (=0) query_cache_type (=0) query_cache_size (> 128M) [see warning above] sort_buffer_size (> 756K) read_rnd_buffer_size (> 1M) join_buffer_size (> 656.0K, or always use indexes with JOINs) tmp_table_size (> 74M) max_heap_table_size (> 74M) innodb_log_file_size should be (=1G) if possible, so InnoDB total log files size equals to 25% of buffer pool size. The server is 2 x 2.4GHz Octa-Core E5-2630 v3 Haswell Xeon, Ram: 96 GB The forum is large, about 5.5 million posts or so. Thank you for reviewing and for any suggestions.
Makoto Posted December 15, 2020 Posted December 15, 2020 Hey there! Are you just looking for suggestions, or are you interested in hiring one of us to perform a detailed tuning of your server and IPS configuration for you? If it's the latter, I'd be happy to offer my services to you. Server tuning usually runs around $300-$600 depending on the amount of work needed, how thorough you want the service to be and the size of the community. If you're interested, just send me a message here!
bfarber Posted December 16, 2020 Posted December 16, 2020 1) Convert MyISAM tables to InnoDB 2) Make sure that your MySQL server is tuned for InnoDB. 3) Follow the general recommendations Quote Run OPTIMIZE TABLE to defragment tables for better performance OPTIMIZE TABLE `enotal1_enotal1`.`ipb_core_edit_history`; -- can free 524.075992584229 MB OPTIMIZE TABLE `enotal1_enotal1`.`ipb_core_output_cache`; -- can free 159 MB OPTIMIZE TABLE `enotal1_enotal1`.`vb_searchlog`; -- can free 305.898475646973 MB Total freed space after theses OPTIMIZE TABLE : 988.974468231201 Mb Quote Variables to adjust: *** MySQL's maximum memory usage is dangerously high *** *** Add RAM before increasing MySQL buffer variables *** query_cache_size (=0) query_cache_type (=0) query_cache_size (> 128M) [see warning above] sort_buffer_size (> 756K) read_rnd_buffer_size (> 1M) join_buffer_size (> 656.0K, or always use indexes with JOINs) tmp_table_size (> 74M) max_heap_table_size (> 74M) innodb_log_file_size should be (=1G) if possible, so InnoDB total log files size equals to 25% of buffer pool size. Beyond that, you haven't really clarified - are you sure MySQL is the problem? There are many web server optimizations that can make a huge difference too. Do you use a CDN of any sort, or reverse proxy caching? SeNioR- and kRSB 2
virap Posted December 16, 2020 Author Posted December 16, 2020 Thank you very much. I am not sure mysql is the problem, simply making sure the configuration is optimized to eliminate mysql as a suspect. Do you think it might be the apache, php configuration? anything specific I should l check for optimal IPB performance? I don't use CDN or proxy caching. There is however, Redis installed and configured with IPB.
bfarber Posted December 17, 2020 Posted December 17, 2020 Optimizing a server is a huge task and it's hard to give a lot of general "do this" recommendations. Most suggestions are specific to an environment. For starters, I'd make sure that (1) you don't have things installed and running that you don't need. The less PHP modules, Apache/nginx modules and so on that are executing, the better off you'll be. And (2) make sure your configurations for Apache/nginx, PHP and MySQL are as optimal as can be. There are thousands of tutorials online that can help you determine what the optimal configuration for your environment and level of traffic should be, and I'd start with reviewing some of those. Basic things like ensuring static files are cached can go a long ways. Makoto 1
Recommended Posts