Jump to content

MySQL Tunning for 3.2.x


Recommended Posts

Posted

[--] Up for: 68d 1h 9m 27s (116M q [19.890 qps], 5M conn, TX: 2732B, RX: 68B)

[--] Reads / Writes: 52% / 48%
[--] Total buffers: 512.0M global + 6.7M per thread (50 max threads)
[OK] Maximum possible memory usage: 846.4M (10% of installed RAM)
[OK] Slow queries: 0% (241/116M)
[OK] Highest usage of available connections: 40% (20/50)
[OK] Key buffer size / total MyISAM indexes: 256.0M/205.4M
[OK] Key buffer hit rate: 100.0% (5B cached / 1M reads)
[OK] Query cache efficiency: 47.1% (32M cached / 68M selects)
[!!] Query cache prunes per day: 5000
[OK] Sorts requiring temporary tables: 0% (269 temp sorts / 2M sorts)
[!!] Joins performed without indexes: 66288
[!!] Temporary tables created on disk: 39% (1M on disk / 3M total)
[OK] Thread cache hit rate: 99% (20 created / 5M connections)
[!!] Table cache hit rate: 1% (435 open / 37K opened)
[OK] Open file limit used: 10% (845/8K)
[OK] Table locks acquired immediately: 99% (104M immediate / 104M locks)


For some reason table cache hit rate is 1%. Should't it be more?


skip-external-locking
default-character-set=utf8
character-set-server = utf8
collation-server = utf8_general_ci
bind-address = 127.0.0.1
key_buffer = 312M
max_allowed_packet = 64M
thread_stack = 192K
thread_cache_size = 128
myisam-recover = BACKUP
max_connections = 50
open_files_limit = 8192
table_cache = 2048
table_definition_cache = 1200
#thread_concurrency = 8
tmp_table_size = 128M
max_heap_table_size = 128M
interactive_timeout = 120
connect_timeout = 15
wait_timeout = 120
innodb_buffer_pool_size = 64M
innodb_log_file_size = 32M
innodb_log_buffer_size = 8M
innodb_flush_log_at_trx_commit = 2
innodb_flush_method = O_DIRECT
innodb_thread_concurrency = 16
innodb_additional_mem_pool_size = 8M
sort_buffer_size = 4M
read_buffer_size = 256K
read_rnd_buffer_size = 256K
join_buffer_size = 2M
myisam_sort_buffer_size = 512M
low_priority_updates=1
concurrent_insert=2
query_cache_limit = 3M
query_cache_size = 128M
query_cache_type = 1

Anybody got suggestion on how to fix it?

Posted

do this in ssh

wget mysqltuner.pl



wait for it to complete.

Then run:

perl mysqltuner.pl



Once it runs please post the FULL output and post it in here for futher updates.
Posted

here is mine any advice will be welcome


mysqltuner

>>  MySQLTuner 1.1.1 - Major Hayden <major@mhtx.net>

>>  Bug reports, feature requests, and downloads at http://mysqltuner.com/

>>  Run with '--help' for additional options and output filtering

Please enter your MySQL administrative login: root

Please enter your MySQL administrative password:

-------- General Statistics --------------------------------------------------

[--] Skipped version check for MySQLTuner script

[OK] Currently running supported MySQL version 5.0.91-log

[OK] Operating on 64-bit architecture

-------- Storage Engine Statistics -------------------------------------------

[--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster

[--] Data in MyISAM tables: 144M (Tables: 306)

[!!] InnoDB is enabled but isn't being used

[!!] Total fragmented tables: 64

-------- Security Recommendations  -------------------------------------------

[!!] User '' has no password set.

[!!] User '' has no password set.

-------- Performance Metrics -------------------------------------------------

[--] Up for: 4d 21h 59m 14s (2M q [5.465 qps], 174K conn, TX: 145B, RX: 894M)

[--] Reads / Writes: 73% / 27%

[--] Total buffers: 58.0M global + 1.6M per thread (100 max threads)

[OK] Maximum possible memory usage: 220.5M (3% of installed RAM)

[OK] Slow queries: 0% (0/2M)

[OK] Highest usage of available connections: 15% (15/100)

[OK] Key buffer size / total MyISAM indexes: 16.0M/30.2M

[OK] Key buffer hit rate: 99.8% (97M cached / 148K reads)

[!!] Query cache is disabled

[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 160K sorts)

[!!] Temporary tables created on disk: 38% (36K on disk / 95K total)

[!!] Thread cache is disabled

[!!] Table cache hit rate: 0% (64 open / 25K opened)

[OK] Open file limit used: 11% (122/1K)

[OK] Table locks acquired immediately: 99% (3M immediate / 3M locks)

-------- Recommendations -----------------------------------------------------

General recommendations:

    Add skip-innodb to MySQL configuration to disable InnoDB

    Run OPTIMIZE TABLE to defragment tables for better performance

    Enable the slow query log to troubleshoot bad queries

    When making adjustments, make tmp_table_size/max_heap_table_size equal

    Reduce your SELECT DISTINCT queries without LIMIT clauses

    Set thread_cache_size to 4 as a starting value

    Increase table_cache gradually to avoid file descriptor limits

Variables to adjust:

    query_cache_size (>= 8M)

    tmp_table_size (> 32M)

    max_heap_table_size (> 16M)

    thread_cache_size (start at 4)

    table_cache (> 64)

Posted

I have 3 IPB forums hosting on same machine and one Joomla website. Website usage is pretty less and was never an issue. Website DB is 2Mb or so i think. All this Joins performed without indexes: 68609 is from IPB.

Thanks


>>  MySQLTuner 1.2.0 - Major Hayden <major@mhtx.net>

>>  Bug reports, feature requests, and downloads at http://mysqltuner.com/

>>  Run with '--help' for additional options and output filtering

-------- General Statistics --------------------------------------------------

[--] Skipped version check for MySQLTuner script

[OK] Currently running supported MySQL version 5.1.58-1~dotdeb.0-log

[OK] Operating on 64-bit architecture


-------- Storage Engine Statistics -------------------------------------------

[--] Status: +Archive -BDB -Federated -InnoDB -ISAM -NDBCluster

[--] Data in MyISAM tables: 824M (Tables: 1273)

[--] Data in MEMORY tables: 1M (Tables: 4)

[!!] Total fragmented tables: 43


-------- Security Recommendations  -------------------------------------------

[OK] All database users have passwords assigned


-------- Performance Metrics -------------------------------------------------

[--] Up for: 69d 14h 34m 29s (120M q [19.994 qps], 5M conn, TX: 2806B, RX: 69B)

[--] Reads / Writes: 52% / 48%

[--] Total buffers: 512.0M global + 6.7M per thread (50 max threads)

[OK] Maximum possible memory usage: 846.4M (10% of installed RAM)

[OK] Slow queries: 0% (279/120M)

[OK] Highest usage of available connections: 40% (20/50)

[OK] Key buffer size / total MyISAM indexes: 256.0M/209.8M

[OK] Key buffer hit rate: 100.0% (5B cached / 1M reads)

[OK] Query cache efficiency: 47.2% (33M cached / 70M selects)

[!!] Query cache prunes per day: 4968

[OK] Sorts requiring temporary tables: 0% (269 temp sorts / 2M sorts)

[!!] Joins performed without indexes: 68609

[!!] Temporary tables created on disk: 39% (1M on disk / 3M total)

[OK] Thread cache hit rate: 99% (20 created / 5M connections)

[!!] Table cache hit rate: 4% (1K open / 39K opened)

[OK] Open file limit used: 36% (2K/8K)

[OK] Table locks acquired immediately: 99% (108M immediate / 108M locks)


-------- Recommendations -----------------------------------------------------

General recommendations:

	Run OPTIMIZE TABLE to defragment tables for better performance

	Adjust your join queries to always utilize indexes

	When making adjustments, make tmp_table_size/max_heap_table_size equal

	Reduce your SELECT DISTINCT queries without LIMIT clauses

	Increase table_cache gradually to avoid file descriptor limits

Variables to adjust:

	query_cache_size (> 128M)

	join_buffer_size (> 2.0M, or always use indexes with joins)

	tmp_table_size (> 128M)

	max_heap_table_size (> 128M)

	table_cache (> 2048)

Posted

I have 3 IPB forums hosting on same machine and one Joomla website. Website usage is pretty less and was never an issue. Website DB is 2Mb or so i think. All this [color=#660066]Joins[/color][color=#000000] performed without indexes[/color][color=#666600]:[/color][color=#000000] [/color][color=#006666]68609 is from IPB. [/color]



Thanks




>>  MySQLTuner 1.2.0 - Major Hayden <major@mhtx.net>

>>  Bug reports, feature requests, and downloads at http://mysqltuner.com/

>>  Run with '--help' for additional options and output filtering

-------- General Statistics --------------------------------------------------

[--] Skipped version check for MySQLTuner script

[OK] Currently running supported MySQL version 5.1.58-1~dotdeb.0-log

[OK] Operating on 64-bit architecture


-------- Storage Engine Statistics -------------------------------------------

[--] Status: +Archive -BDB -Federated -InnoDB -ISAM -NDBCluster

[--] Data in MyISAM tables: 824M (Tables: 1273)

[--] Data in MEMORY tables: 1M (Tables: 4)

[!!] Total fragmented tables: 43


-------- Security Recommendations  -------------------------------------------

[OK] All database users have passwords assigned


-------- Performance Metrics -------------------------------------------------

[--] Up for: 69d 14h 34m 29s (120M q [19.994 qps], 5M conn, TX: 2806B, RX: 69B)

[--] Reads / Writes: 52% / 48%

[--] Total buffers: 512.0M global + 6.7M per thread (50 max threads)

[OK] Maximum possible memory usage: 846.4M (10% of installed RAM)

[OK] Slow queries: 0% (279/120M)

[OK] Highest usage of available connections: 40% (20/50)

[OK] Key buffer size / total MyISAM indexes: 256.0M/209.8M

[OK] Key buffer hit rate: 100.0% (5B cached / 1M reads)

[OK] Query cache efficiency: 47.2% (33M cached / 70M selects)

[!!] Query cache prunes per day: 4968

[OK] Sorts requiring temporary tables: 0% (269 temp sorts / 2M sorts)

[!!] Joins performed without indexes: 68609

[!!] Temporary tables created on disk: 39% (1M on disk / 3M total)

[OK] Thread cache hit rate: 99% (20 created / 5M connections)

[!!] Table cache hit rate: 4% (1K open / 39K opened)

[OK] Open file limit used: 36% (2K/8K)

[OK] Table locks acquired immediately: 99% (108M immediate / 108M locks)


-------- Recommendations -----------------------------------------------------

General recommendations:

	Run OPTIMIZE TABLE to defragment tables for better performance

	Adjust your join queries to always utilize indexes

	When making adjustments, make tmp_table_size/max_heap_table_size equal

	Reduce your SELECT DISTINCT queries without LIMIT clauses

	Increase table_cache gradually to avoid file descriptor limits

Variables to adjust:

	query_cache_size (> 128M)

	join_buffer_size (> 2.0M, or always use indexes with joins)

	tmp_table_size (> 128M)

	max_heap_table_size (> 128M)

	table_cache (> 2048)




make your my.cnf look like this:



[mysqld]


skip-external-locking


default-character-set=utf8


character-set-server = utf8


collation-server = utf8_general_ci


bind-address = 127.0.0.1


key_buffer = 512M


max_allowed_packet = 32M


thread_stack = 192K


thread_cache_size = 128


myisam-recover = BACKUP


max_connections = 50


open_files_limit = 8192


table_cache = 3086


table_definition_cache = 1200


tmp_table_size = 160M


max_heap_table_size = 160M


interactive_timeout = 90


connect_timeout = 90


wait_timeout = 90


innodb_buffer_pool_size = 64M


innodb_log_file_size = 32M


innodb_log_buffer_size = 8M


innodb_flush_log_at_trx_commit = 2


innodb_flush_method = O_DIRECT


innodb_thread_concurrency = 16


innodb_additional_mem_pool_size = 8M


sort_buffer_size = 4M


read_buffer_size = 256K


read_rnd_buffer_size = 256K


join_buffer_size = 3M


myisam_sort_buffer_size = 512M


low_priority_updates=1


concurrent_insert=2


query_cache_limit = 3M


query_cache_size = 128M


query_cache_type = 1



then restart your mysql:

/etc/init.d/mysqld restart



Then wait 24 hours and post the output of your tuner.
Posted

Gary, how do you find join perform without indexes? I guess those queries are performance killer?


[!!] Joins performed without indexes: 849

Posted

Gary, how do you find join perform without indexes? I guess those queries are performance killer?




[!!] Joins performed without indexes: 849


Rather than trying to make joins without indexes, you can add indexes. See what the joins are made on and add the appropriate indexes in your mysql.
Posted

Rather than trying to make joins without indexes, you can add indexes. See what the joins are made on and add the appropriate indexes in your mysql.




Is it good to add indexes? From 3.2.x I see high amount of joins without indexes. I thought join also slow down inserts. So if anyone has experience, I would need some help.

How do you find queries not using join?

Thanks
Posted

Is it good to add indexes? From 3.2.x I see high amount of joins without indexes. I thought join also slow down inserts. So if anyone has experience, I would need some help.



The amount of "slow down" caused by additional indexes are mostly insignificant. The true "down side" of additional indexes are space requirements. But I find that to be quite insignificant too in relative to the size of the entire database (unless you're making indexes on everything and more...). If you believe that the column is frequently looked up by it's value, it's worth it to add an index.
But that doesn't mean you should add indexes for everything. The necessity of indexes on joins are also directional. The condition of the table that's being joined is the one that'll benefit from an insert, not the other way around.

How do you find queries not using join?



I'm not too sure what you're asking here.
There are many ways to rewrite the query to not use joins, depending on the query itself. But joins are one of the most efficient way to express an idea. If you separate it into multiple queries, for example, it's FAR FAR worse. If you make single queries with inner selects, etc, it's roughly 3x slower (on some fairly large test data I had... lol).
Posted

I never had this many joins without indexes in 3.1.x and it all started with 3.2.x.

Within a day, I see 2000 joins with indexes.. I just want to figure out if it's IPB 3.2.x or some mod causing it..

Posted

I never had this many joins without indexes in 3.1.x and it all started with 3.2.x.



Within a day, I see 2000 joins with indexes.. I just want to figure out if it's IPB 3.2.x or some mod causing it..


I'd bet on your mod...
Posted

How to find which one?



You can try running a profiler on a testing server. (Running profiler on live server will very easy kill your server as it'll take like 10x more resources). (fastest if you already have it setup...)
You can just try code review. (good if you know programming)
Or you can just try disabling some of them and see if it helps and isolate it. (i'd say last resort...)
Posted

Hello,

I'm using MariaDB 5.3 wich "radically improves performance for subqueries as well as for joins and single-table queries over large data set2", quoted from they site.
I haven't got results to prove any difference from previous setup with mysql, but you can try by your self.

Posted

Here's mine. With IPB ignore the "Table cache hit rate". The way IPB database is setup it will always increment high. No matter how much you increase your table cache. Set table cache to # of tables plus round number. Example if you have 211 tables set to 256. Just my setup. Go with what you think is best :smile:


-------- General Statistics --------------------------------------------------															  

[--] Skipped version check for MySQLTuner script																							

[OK] Currently running supported MySQL version 5.1.41-3ubuntu12.8-log																		

[OK] Operating on 32-bit architecture with less than 2GB RAM																				


-------- Storage Engine Statistics -------------------------------------------															  

[--] Status: -Archive -BDB -Federated -InnoDB -ISAM -NDBCluster																			  

[--] Data in MyISAM tables: 29M (Tables: 227)																								

[!!] Total fragmented tables: 32																											


-------- Performance Metrics -------------------------------------------------															  

[--] Up for: 14h 41m 0s (110K q [2.092 qps], 10K conn, TX: 486M, RX: 41M)																	

[--] Reads / Writes: 46% / 54%																											  

[--] Total buffers: 48.0M global + 2.8M per thread (50 max threads)																		  

[OK] Maximum possible memory usage: 188.6M (29% of installed RAM)																			

[OK] Slow queries: 0% (0/110K)																											  

[OK] Highest usage of available connections: 14% (7/50)																					  

[OK] Key buffer size / total MyISAM indexes: 16.0M/5.4M																					  

[OK] Key buffer hit rate: 99.8% (1M cached / 1K reads)																					  

[OK] Query cache efficiency: 42.7% (23K cached / 53K selects)																				

[OK] Query cache prunes per day: 0																										  

[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 2K sorts)																		  

[!!] Temporary tables created on disk: 26% (160 on disk / 608 total)																		

[OK] Thread cache hit rate: 99% (7 created / 10K connections)																				

[OK] Table cache hit rate: 23% (254 open / 1K opened)																						

[OK] Open file limit used: 49% (507/1K)																									  

[OK] Table locks acquired immediately: 99% (93K immediate / 93K locks)



Also I prefer tuning-primer.sh over mysqltuner

Posted

I'd bet on your mod...




No, it was IP.SEO and Inline notifications. Deleted all error logs, Search Engine logs and lowered the limit of inline notifications to 50 and forum is faster again.



SELECT /*!40001 SQL_NO_CACHE */ * FROM `ipb_spider_logs`;

# Time: 111030  8:00:15

# User@Host: gamingforums[gamingforums] @ localhost []

# Query_time: 3.203056  Lock_time: 0.000000 Rows_sent: 1277760  Rows_examined: 1277760

SET timestamp=1319958015;

SELECT /*!40001 SQL_NO_CACHE */ * FROM `ipb_spider_logs`;

# Time: 111030  9:00:15

# User@Host: gamingforums[gamingforums] @ localhost []

# Query_time: 3.288256  Lock_time: 0.000000 Rows_sent: 1279498  Rows_examined: 1279498

SET timestamp=1319961615;

SELECT /*!40001 SQL_NO_CACHE */ * FROM `ipb_spider_logs`;

# Time: 111030 10:00:15

# User@Host: gamingforums[gamingforums] @ localhost []

# Query_time: 3.325171  Lock_time: 0.000000 Rows_sent: 1281379  Rows_examined: 1281379

SET timestamp=1319965215;

SELECT /*!40001 SQL_NO_CACHE */ * FROM `ipb_spider_logs`;

# Time: 111030 11:00:15

# User@Host: gamingforums[gamingforums] @ localhost []

# Query_time: 3.348546  Lock_time: 0.000000 Rows_sent: 1283179  Rows_examined: 1283179

SET timestamp=1319968815;

SELECT /*!40001 SQL_NO_CACHE */ * FROM `ipb_spider_logs`;

# Time: 111030 12:00:14

# User@Host: gamingforums[gamingforums] @ localhost []

# Query_time: 3.206847  Lock_time: 0.000000 Rows_sent: 1284799  Rows_examined: 1284799

SET timestamp=1319972414;

SELECT /*!40001 SQL_NO_CACHE */ * FROM `ipb_spider_logs`;

# Time: 111030 13:00:15

# User@Host: gamingforums[gamingforums] @ localhost []

# Query_time: 3.238244  Lock_time: 0.000000 Rows_sent: 1286497  Rows_examined: 1286497

SET timestamp=1319976015;

SELECT /*!40001 SQL_NO_CACHE */ * FROM `ipb_spider_logs`;

# Time: 111030 14:00:07

# User@Host: gamingforums[gamingforums] @ localhost []

# Query_time: 3.241782  Lock_time: 0.000000 Rows_sent: 168886  Rows_examined: 168886

SET timestamp=1319979607;

SELECT /*!40001 SQL_NO_CACHE */ * FROM `ipb_inline_notifications`;

# Time: 111030 14:00:17

# User@Host: gamingforums[gamingforums] @ localhost []

# Query_time: 4.429939  Lock_time: 0.000000 Rows_sent: 1288391  Rows_examined: 1288391

SET timestamp=1319979617;

SELECT /*!40001 SQL_NO_CACHE */ * FROM `ipb_spider_logs`;



Now next step is to find queries not using joins but at least this longer time taking queries are fixed.

Posted

What do you mean by inline notifications?




When someone quotes your post, reply to your topic you get notifications which is show next to PM box on top right corner. That one.
Posted

When someone quotes your post, reply to your topic you get notifications which is show next to PM box on top right corner. That one.




So what did you do about the SEO slowing things down?
Posted

For me inline notifications slowed down the site because it would process the emails subscriptions before the reply is made. So if there's 50 emails to send out its a lot slower.

  • 2 weeks later...
Posted

I'm considering switching hosting because of the performance issues, but if we can clean it up, that would be great:

>> MySQLTuner 1.2.0 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering

-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.0.77
[!!] Switch to 64-bit OS - MySQL cannot currently use all of your RAM

-------- Storage Engine Statistics -------------------------------------------
[--] Status: -Archive +BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in MyISAM tables: 883M (Tables: 489)
[!!] InnoDB is enabled but isn't being used
[!!] BDB is enabled but isn't being used
[!!] Total fragmented tables: 22

-------- Security Recommendations -------------------------------------------
[OK] All database users have passwords assigned

-------- Performance Metrics -------------------------------------------------
[--] Up for: 141d 21h 29m 18s (619M q [50.559 qps], 17M conn, TX: 3B, RX: 986M)
[--] Reads / Writes: 55% / 45%
[--] Total buffers: 82.0M global + 18.4M per thread (300 max threads)
[!!] Allocating > 2GB RAM on 32-bit systems can cause system instability
[!!] Maximum possible memory usage: 5.5G (153% of installed RAM)
[OK] Slow queries: 0% (3K/619M)
[OK] Highest usage of available connections: 40% (120/300)
[OK] Key buffer size / total MyISAM indexes: 8.0M/396.7M
[OK] Key buffer hit rate: 95.9% (160B cached / 6B reads)
[OK] Query cache efficiency: 76.3% (361M cached / 474M selects)
[!!] Query cache prunes per day: 86383
[OK] Sorts requiring temporary tables: 8% (645K temp sorts / 7M sorts)
[!!] Joins performed without indexes: 288221
[OK] Temporary tables created on disk: 7% (1M on disk / 16M total)
[OK] Thread cache hit rate: 99% (120 created / 17M connections)
[!!] Table cache hit rate: 13% (7K open / 53K opened)
[OK] Open file limit used: 16% (10K/65K)
[OK] Table locks acquired immediately: 99% (254M immediate / 256M locks)

-------- Recommendations -----------------------------------------------------
General recommendations:
Add skip-innodb to MySQL configuration to disable InnoDB
Add skip-bdb to MySQL configuration to disable BDB
Run OPTIMIZE TABLE to defragment tables for better performance
Enable the slow query log to troubleshoot bad queries
Adjust your join queries to always utilize indexes
Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
*** MySQL's maximum memory usage is dangerously high ***
*** Add RAM before increasing MySQL buffer variables ***
query_cache_size (> 32M)
join_buffer_size (> 8.0M, or always use indexes with joins)
table_cache (> 32612)


/etc/my.cnf:
[mysqld]
socket = /var/lib/mysql/mysql.sock
safe-show-database
old-password=1
max_connections=500
interactive_timeout=100
wait_timeout=30
connect_timeout=30
thread_cache_size=128
key_buffer=64M
join_buffer=8M
max_allowed_packet=32M
table_cache=1024
sort_buffer=32M
record_buffer=8M
thread_cache_size=128
max_user_connections=30
thread_concurrency=8
myisam_sort_buffer_size=64M
query_cache_limit=2M
query_cache_size=32M
query_cache_type=1
tmp_table_size=64M
max_heap_table_size=32M
old-passwords = 1
set-variable = net_buffer_length=16K
set-variable = max_allowed_packet=1M
set-variable = myisam_sort_buffer_size=8M
set-variable = sort_buffer=2M
set-variable = max_connections=300
set-variable = key_buffer=8M
set-variable = table_cache=98304

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

Posted

@Ambar:
Try below:
query_cache_size = 128M
sort_buffer = 2M
key_buffer = 256M

I have this problem too with 3.2.x.
[!!] Joins performed without indexes: 288221

Archived

This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...