Jump to content

Need help getting CycleChat up to speed


Recommended Posts

  • Replies 101
  • Created
  • Last Reply

I've been looking in the server optimisation forums and came across this site - http://hoitajat.net/foorumi/ - look how fast it loads in comparison to mine ... if I could get CC going that fast I'd be more than happy !!! :thumbsup:



Cheers,


Shaun :D



Your site is as fast as that...

Maybe you've got something on your computer slower your browser down ?
Link to comment
Share on other sites

Hey Shaun,

I went to both sites, and clicked around from thread to thread and from the main page back to different threads and I have to say I liked the vB site better.

Not just because it seemed faster, but because of the style. If your users were used to a 'clean', white style and now you're giving them 'bulky' blue style I could see why they would think it's slower.

There is nothing wrong with your blue style, if I were a new user and didn't know about, or had used your vB style.

I think if you created a style for IPB that better matched your old vB clean style, your users would love it.

Just my two cents.. :)

Jamie

Link to comment
Share on other sites

Thanks for the feedback.

I wonder if it is the layout that is the issue with people - rather than speed (although I still want to make sure the server is optimised before weighing in to redesigning a new skin and I know the old site loaded and displayed faster - I used the thing for years before converting).

I've changed the Debug Level to 1 to print the queries and load time at the bottom of each page - not sure whether the numbers are high / low or good / bad - maybe someone who knows about these things can advise?

I get different speed results for the new site depending on whether I'm at home on the Win 7 laptop with 4GB, on my day-job XP office PC (IE7 and really sluggish when scrolling forum home pages [lists of threads]), on my XP gaming PC at home (also with IE7 and v.slow scrolling / browsing but then I don't need an up-to-date browser for COD :wink:) and my own XP office PC with FF3.[not sure!]

All of these machines ran the "old" site without any speed issues or browsing differences - it was just the same on all of them. The new site is different on virtually all of them - the fastest being the Win 7 laptop with IE8.

I've run the site for years and when you've done something for that long you know when something is not right, and I know it's not right at the moment, so I suppose I've got to start from the ground up and get the server optimised first and then look at a possible look-a-like skin to temp people back who feel they cannot get on with the "new" one.

Thanks,
Shaun :D

Link to comment
Share on other sites


Your site is as fast as that...



Maybe you've got something on your computer slower your browser down ?





I agree. All 3 of those sites (vB one included) load topics, forums and the index page at speeds which are not noticeably different to me. All of them load in under a second.
Link to comment
Share on other sites

Okay, thanks for the feedback everyone.

Can a mod please move this to the server optimisation forum?

I'll disucss my set-up there to ensure I've got the hardware running properly, then take a look at possibly creating a look-a-like skin to match the "old" site to see if that helps.

Cheers,
Shaun :D

Link to comment
Share on other sites

I hope so too ...

I've twiddled with my.cnf again tonight and reduced the keepalive time in httpd.conf so we'll see how it's going tomorrow during the day when a few people are on?

To ensure I'm getting the most out of the optimisation exercise what sort of info do I need to post here?

Cheers,
Shaun :D

Link to comment
Share on other sites

I just wanted to point out that the reason your site may feel slower is simply the fact that your skin is much more complex and nicer than your old one. This one uses more images and everything, whereas the other one didn't.

I found that caching php files for a minuet on my server helped speed it up a tony bit, but there isn't really a need to do that, especially on forums :P

Link to comment
Share on other sites


Old forum ~ Currently Active Users: 1 (0 members and 1 guests)


New forum ~ 233 active user(s) (in the past 15 minutes)



Just a thought, might possibly have some effect on load times?




Unfortunately that isn't the case. Even with 500+ users online the vB site never faltered.

I suspect the PHP scripts are slower/more complex and the templates (and resulting parsed pages) are more bulky. IPB 3 utilises CSS and AJAX to a greater degree, and whilst this is desirable for modern forward-moving applications, it does mean they take more processing than there 3-5 year old cousins.

Look at IPB 2 series. Look at how many people have said their sites are slow since upgrading.

vB 3 used CSS to less of a degree. It didn't provide the skinning options that IPB 3 does, but at the same time uses a lot less HTML to render similar views.

There are a lot of elements that go into optimising your forums, and I'd really appreciate some help to make sure my server is doing what it should optimally, then I can start looking at making a less-fancy skin that just gives people the basics they were used to.

Looking good is great, but it's useless if you end up with a deserted forum!!!

Cheers,
Shaun :D
Link to comment
Share on other sites

A forum is about people being able to interact quickly and simply and I'd rather have a busy forum running old software, than a flashy new one that people don't like using.


Curious, why haven't you enabled Fast Reply?

Site seems fairly quick to me, but yes I agree it could be faster. The biggest change you can make to my.cnf is key_buffer_size, with 4GB RAM you could safely set it at 1GB, but there are a couple good tools to help you optimize my.cnf. Tuning-primer.sh is my personal fav, but MySQLTuner is good too. Probably your best path to performance improvement.

You have VERY active members. Only 12,638 members and 1,376,104 posts. That's over a 100 post per user average. I agree the new skin looks better, but it was a lot of change at once. You might have been better served to make IPB look more like the old skin. At a minimum offer the old skin as an option (or even the default) for existing members "CycleChat Classic" while offering the new skin to visitors (new members).

If you've tried everything and still want more performance out of the same hardware, look at adding a CDN (content delivery network), or alternative web server (like Litespeed). However, I'll warn you neither option is cheap.

By the way, you keep mentioning the old board was vB... but it sure looks like PHPBB to me. ;)
Link to comment
Share on other sites

I just registered to view the debug settings. Page load time for the index is actually very good @ 0.1558. It doesn't show a server load. This is because you need to enter a value here: Tools & Settings > System Settings > System, CPU Saving & Optimization. Enter a value of 100 if you want. Any value will cause the debugging to offer a load number instead of "--".

Just a comment, while registering you had reCaptcha plus a challenge question. You really only need one (I'd recommend the challenge question). However, the challenge question I got was ambiguous. Was I supposed to type a number or spell it?

If you have four apples and you eat one, how many have you got left?

Link to comment
Share on other sites

I just noticed this thread:

Have you tried deleting your CC cookie (click here) - not sure why but it can sort all sorts of problems.


Are you running 3.1.2?

Might want to check out this: http://community.invisionpower.com/resources/articles.html/_/ipboard-3x/tips-and-tricks/excessive-server-loads-in-ipb3-r491
Link to comment
Share on other sites


Curious, why haven't you enabled Fast Reply?




I have for registered members.


Site seems fairly quick to me, but yes I agree it could be faster. The biggest change you can make to my.cnf is key_buffer_size, with 4GB RAM you could safely set it at 1GB, but there are a couple good tools to help you optimize my.cnf. Tuning-primer.sh is my personal fav, but MySQLTuner is good too. Probably your best path to performance improvement.




I've been using both of those to help with MySQL but I'll bump up the buffer and see what happens, thanks.


You have VERY active members. Only 12,638 members and 1,376,104 posts. That's over a 100 post per user average. I agree the new skin looks better, but it was a lot of change at once. You might have been better served to make IPB look more like the old skin. At a minimum offer the old skin as an option (or even the default) for existing members "CycleChat Classic" while offering the new skin to visitors (new members).




I'm wondering if that is part of the issue - although I did give everyone the chance to see it and comment on it before moving, but I suppose until you actually get started on using something new you can't truely decide whether it "works" for you or not.

I'm going to get the server running as best I can, then look at getting a skin together that mimics the old vB site.


If you've tried everything and still want more performance out of the same hardware, look at adding a CDN (content delivery network), or alternative web server (like Litespeed). However, I'll warn you neither option is cheap.




I've already emptied my Piggy Bank investing in IPB ... :wink:


By the way, you keep mentioning the old board was vB... but it sure looks like PHPBB to me. ;)




Ah, well, that may be because I moved a lot of the original PhpBB icons over to the vB install when we first upgraded ... :D

Cheers,
Shaun :D
Link to comment
Share on other sites




Just a comment, while registering you had reCaptcha plus a challenge question. You really only need one (I'd recommend the challenge question). However, the challenge question I got was ambiguous. Was I supposed to type a number or spell it?




Thanks for the tip - done that and server load now displaying.

The challenge question accepts both 3 and three, but I suppose it could be better phrased.

Cheers,
Shaun :D
Link to comment
Share on other sites


I just noticed this thread:




I had a couple of issues with cookies when I first moved over, but nothing major since. I've set the Cookie Domain though (I'd originally left it blank because I used several domains for the site, but IPB 3 can only run from a single domain so I may as well set this value!!).

Cheers,
Shaun :D
Link to comment
Share on other sites

Okay, just to set the ball rolling ... I'm using LAMP and have 4GB in my dedicated server.

Here's my.cnf:


[client]


port = 3306


socket = /var/run/mysqld/mysqld.sock



[safe_mysqld]


err-log = /var/log/mysql/mysql.err



[mysqld]


user = mysql


pid-file = /var/run/mysqld/mysqld.pid


socket = /var/run/mysqld/mysqld.sock


port = 3306


max_connections = 300


basedir = /usr


datadir = /var/lib/mysql


tmpdir = /tmp


language = /usr/share/mysql/english


ft_min_word_len = 3



skip-locking


skip-innodb



wait_timeout = 600


key_buffer = 512M


join_buffer_size = 1M


max_allowed_packet = 32M


table_cache = 2800


sort_buffer_size = 2M


read_buffer_size = 2M


read_rnd_buffer_size = 2M


myisam_sort_buffer_size = 128M


thread_cache_size = 128


query_cache_limit = 64M


query_cache_size = 64M


query_cache_type = 1


tmp_table_size = 256M


max_heap_table_size = 256M



log-slow-queries = /var/log/mysql-slow.log


long_query_time = 2




[mysqldump]


quick


max_allowed_packet = 16M



[mysql]



[isamchk]


key_buffer = 64M


sort_buffer_size = 64M


read_buffer = 16M


write_buffer = 16M


ft_min_word_len = 3



[myisamchk]


key_buffer = 64M


sort_buffer_size = 64M


read_buffer = 16M


write_buffer = 16M


ft_min_word_len = 3





Here's the result from mysqltuner.pl:


-------- General Statistics --------------------------------------------------


[--] Skipped version check for MySQLTuner script


[OK] Currently running supported MySQL version 5.0.51a-24+lenny2-log


[!!] Switch to 64-bit OS - MySQL cannot currently use all of your RAM



-------- Storage Engine Statistics -------------------------------------------


[--] Status: +Archive -BDB -Federated -InnoDB -ISAM -NDBCluster


[--] Data in MyISAM tables: 3G (Tables: 1137)


[--] Data in MEMORY tables: 7M (Tables: 5)


[!!] Total fragmented tables: 29



-------- Performance Metrics -------------------------------------------------


[--] Up for: 1d 13h 14m 53s (5M q [39.827 qps], 197K conn, TX: 3B, RX: 1B)


[--] Reads / Writes: 49% / 51%


[--] Total buffers: 842.0M global + 7.2M per thread (300 max threads)


[!!] Allocating > 2GB RAM on 32-bit systems can cause system instability


[!!] Maximum possible memory usage: 2.9G (75% of installed RAM)


[OK] Slow queries: 0% (40/5M)


[OK] Highest usage of available connections: 8% (24/300)


[OK] Key buffer size / total MyISAM indexes: 512.0M/2.3G


[OK] Key buffer hit rate: 99.8% (83M cached / 127K reads)


[OK] Query cache efficiency: 55.4% (1M cached / 3M selects)


[!!] Query cache prunes per day: 22861


[OK] Sorts requiring temporary tables: 1% (333 temp sorts / 17K sorts)


[!!] Temporary tables created on disk: 35% (5K on disk / 15K total)


[OK] Thread cache hit rate: 99% (24 created / 197K connections)


[OK] Table cache hit rate: 83% (2K open / 2K opened)


[OK] Open file limit used: 59% (3K/5K)


[OK] Table locks acquired immediately: 99% (5M immediate / 5M locks)



-------- Recommendations -----------------------------------------------------


General recommendations:


Run OPTIMIZE TABLE to defragment tables for better performance


Temporary table size is already large - reduce result set size


Reduce your SELECT DISTINCT queries without LIMIT clauses


Variables to adjust:


query_cache_size (> 64M)





Here's the result of tuning-primer.sh:


-- MYSQL PERFORMANCE TUNING PRIMER --


- By: Matthew Montgomery -



MySQL Version 5.0.51a-24+lenny2-log i486



Uptime = 1 days 13 hrs 15 min 32 sec


Avg. qps = 39


Total Questions = 5343192


Threads Connected = 2



Warning: Server has not been running for at least 48hrs.


It may not be safe to use these recommendations



To find out more information on how each of these


runtime variables effects performance visit:


http://dev.mysql.com/doc/refman/5.0/en/server-system-variables.html


Visit http://www.mysql.com/products/enterprise/advisors.html


for info about MySQL's Enterprise Monitoring and Advisory Service



SLOW QUERIES


The slow query log is enabled.


Current long_query_time = 2 sec.


You have 40 out of 5343220 that take longer than 2 sec. to complete


Your long_query_time seems to be fine



BINARY UPDATE LOG


The binary update log is NOT enabled.


You will not be able to do point in time recovery


See http://dev.mysql.com/doc/refman/5.0/en/point-in-time-recovery.html



WORKER THREADS


Current thread_cache_size = 128


Current threads_cached = 22


Current threads_per_sec = 0


Historic threads_per_sec = 0


Your thread_cache_size is fine



MAX CONNECTIONS


Current max_connections = 300


Current threads_connected = 3


Historic max_used_connections = 24


The number of used connections is 8% of the configured maximum.


You are using less than 10% of your configured max_connections.


Lowering max_connections could help to avoid an over-allocation of memory


See "MEMORY USAGE" section to make sure you are not over-allocating



No InnoDB Support Enabled!



MEMORY USAGE


Max Memory Ever Allocated : 758 M


Configured Max Per-thread Buffers : 2.10 G


Configured Max Global Buffers : 586 M


Configured Max Memory Limit : 2.67 G


Physical Memory : 3.89 G


Max memory limit seem to be within acceptable norms



KEY BUFFER


Current MyISAM index space = 2.33 G


Current key_buffer_size = 512 M


Key cache miss rate is 1 : 657


Key buffer free ratio = 67 %


Your key_buffer_size seems to be fine



QUERY CACHE


Query cache is enabled


Current query_cache_size = 64 M


Current query_cache_used = 16 M


Current query_cache_limit = 64 M


Current Query cache Memory fill ratio = 25.38 %


Current query_cache_min_res_unit = 4 K


Your query_cache_size seems to be too high.


Perhaps you can use these resources elsewhere


MySQL won't cache query results that are larger than query_cache_limit in size



SORT OPERATIONS


Current sort_buffer_size = 2 M


Current read_rnd_buffer_size = 1 M


Sort buffer seems to be fine



JOINS


Current join_buffer_size = 1.00 M


You have had 27 queries where a join could not use an index properly


You should enable "log-queries-not-using-indexes"


Then look for non indexed joins in the slow query log.


If you are unable to optimize your queries you may want to increase your


join_buffer_size to accommodate larger joins in one pass.



Note! This script will still suggest raising the join_buffer_size when


ANY joins not using indexes are found.



OPEN FILES LIMIT


Current open_files_limit = 5910 files


The open_files_limit should typically be set to at least 2x-3x


that of table_cache if you have heavy MyISAM usage.


Your open_files_limit value seems to be fine



TABLE CACHE


Current table_cache value = 2800 tables


You have a total of 1159 tables


You have 2369 open tables.


The table_cache value seems to be fine



TEMP TABLES


Current max_heap_table_size = 256 M


Current tmp_table_size = 256 M


Of 9960 temp tables, 35% were created on disk


Perhaps you should increase your tmp_table_size and/or max_heap_table_size


to reduce the number of disk-based temporary tables


Note! BLOB and TEXT columns are not allow in memory tables.


If you are using these columns raising these values might not impact your


ratio of on disk temp tables.



TABLE SCANS


Current read_buffer_size = 1 M


Current table scan ratio = 62 : 1


read_buffer_size seems to be fine



TABLE LOCKING


Current Lock Wait ratio = 1 : 335


You may benefit from selective use of InnoDB.


If you have long running SELECT's against MyISAM tables and perform


frequent updates consider setting 'low_priority_updates=1'


If you have a high concurrency of inserts on Dynamic row-length tables


consider setting 'concurrent_insert=2'.





... and finally, here's a snapshot of top with my current board stats (91 members, 120 guests):


top - 11:46:49 up 12 days, 21:43, 1 user, load average: 1.15, 1.18, 1.00


Tasks: 145 total, 6 running, 139 sleeping, 0 stopped, 0 zombie


Cpu(s): 23.2%us, 1.5%sy, 0.0%ni, 75.2%id, 0.0%wa, 0.0%hi, 0.2%si, 0.0%st


Mem: 4087252k total, 3477496k used, 609756k free, 196868k buffers


Swap: 1951888k total, 616k used, 1951272k free, 2417288k cached



PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND


29142 mysql 20 0 644m 275m 5088 S 3 6.9 68:19.48 mysqld


3881 root 20 0 99.5m 92m 91m S 0 2.3 1:10.19 searchd


2971 clamav 20 0 88816 76m 844 S 0 1.9 0:01.80 clamd


3425 www-data 20 0 52340 30m 4440 S 0 0.8 0:02.16 apache2


4814 www-data 20 0 47912 26m 4564 S 15 0.7 0:04.54 apache2


13875 nobody 20 0 27608 25m 684 S 0 0.6 8:17.01 memcached


5076 www-data 20 0 47164 25m 4620 S 0 0.6 0:01.74 apache2


4699 www-data 20 0 45360 23m 4596 S 7 0.6 0:05.28 apache2


5090 www-data 20 0 45292 23m 4488 S 0 0.6 0:01.10 apache2


5091 www-data 20 0 44992 23m 4488 S 0 0.6 0:02.56 apache2


5086 www-data 20 0 44768 22m 4444 R 5 0.6 0:01.56 apache2


4826 www-data 20 0 43584 21m 4596 S 0 0.5 0:03.36 apache2


5088 www-data 20 0 43528 21m 4444 S 0 0.5 0:00.58 apache2


4825 www-data 20 0 43276 21m 4620 S 0 0.5 0:04.90 apache2


4822 www-data 20 0 43296 21m 4640 S 4 0.5 0:03.32 apache2


5062 www-data 20 0 42432 20m 4424 S 0 0.5 0:00.36 apache2


4827 www-data 20 0 41920 20m 4612 S 0 0.5 0:04.46 apache2


4805 www-data 20 0 41896 20m 4532 S 6 0.5 0:02.70 apache2


3180 www-data 20 0 41732 20m 4620 S 0 0.5 0:11.64 apache2


4395 www-data 20 0 41580 19m 4604 S 0 0.5 0:08.34 apache2


5140 www-data 20 0 41608 19m 4480 S 0 0.5 0:01.38 apache2


4812 www-data 20 0 41428 19m 4588 R 7 0.5 0:02.42 apache2


5080 www-data 20 0 41348 19m 4516 S 0 0.5 0:01.84 apache2


5075 www-data 20 0 41320 19m 4516 S 0 0.5 0:00.88 apache2


4811 www-data 20 0 40908 19m 4592 S 0 0.5 0:04.52 apache2


2418 www-data 20 0 40772 19m 4624 S 0 0.5 0:12.40 apache2


4697 www-data 20 0 40580 18m 4544 S 0 0.5 0:04.36 apache2


4806 www-data 20 0 39320 17m 4596 S 0 0.4 0:03.84 apache2


5078 www-data 20 0 39176 17m 4444 S 0 0.4 0:01.30 apache2


5008 www-data 20 0 37672 15m 4524 S 0 0.4 0:02.50 apache2


4819 www-data 20 0 37496 15m 4524 S 0 0.4 0:01.98 apache2


5221 root 20 0 33296 14m 6456 S 0 0.4 0:22.82 apache2


5188 www-data 20 0 33684 11m 3064 S 0 0.3 0:00.00 apache2


5186 www-data 20 0 33296 9976 1560 S 0 0.2 0:00.00 apache2


2712 bind 20 0 50664 9612 2292 S 0 0.2 0:00.24 named


2747 root 20 0 50268 9340 2304 S 0 0.2 0:00.08 named


3079 root 20 0 12480 8616 1736 S 0 0.2 0:10.04 miniserv.pl


7760 www-data 20 0 25924 7356 468 S 0 0.2 0:00.10 apache2


3486 root 20 0 39880 4184 1752 R 0 0.1 15:46.16 fail2ban-server


2172 root 20 0 8212 2532 2060 R 0 0.1 0:00.06 sshd


2185 root 20 0 4208 1640 1308 S 0 0.0 0:00.20 bash


29109 root 20 0 2848 1336 1088 S 0 0.0 0:00.00 mysqld_safe


3779 root 20 0 4684 1280 736 S 0 0.0 0:32.82 monit


5205 qmaild 20 0 4868 1228 988 S 0 0.0 0:00.00 qmail-smtpd


5165 qmaild 20 0 4868 1224 988 S 0 0.0 0:00.00 qmail-smtpd


4767 qmaild 20 0 4868 1220 984 S 0 0.0 0:00.00 qmail-smtpd


4952 qmaild 20 0 4868 1220 984 S 0 0.0 0:00.00 qmail-smtpd


4953 qmaild 20 0 4868 1220 984 S 0 0.0 0:00.00 qmail-smtpd


5155 qmaild 20 0 4868 1220 984 S 0 0.0 0:00.00 qmail-smtpd


5175 qmaild 20 0 4868 1220 984 S 0 0.0 0:00.00 qmail-smtpd


4999 qmaild 20 0 4868 1216 984 S 0 0.0 0:00.00 qmail-smtpd


5101 qmaild 20 0 4868 1216 984 S 0 0.0 0:00.00 qmail-smtpd


5077 qmaild 20 0 4868 1188 948 S 0 0.0 0:00.00 qmail-smtpd




Link to comment
Share on other sites

Your key_buffer_size looks good. Like I said, that stands to make the most difference.

I run a similar LAMP/4GB setup, I've added my settings after yours. I've been tweaking for years, but I'm sure they could still stand improvement. Every server is different:
wait_timeout = 600 // mine is 100
// + connect_timeout = 20
// + interactive_timeout = 1200
key_buffer = 512M // mine is 1000M, but a larger db
join_buffer_size = 1M // 4M
max_allowed_packet = 32M // 16M
table_cache = 2800 // 2500
sort_buffer_size = 2M // 1M
read_buffer_size = 2M // 256K (read about a bug in mysql 5, anything over 256K can slow).
read_rnd_buffer_size = 2M // null
myisam_sort_buffer_size = 128M // 256M
thread_cache_size = 128 // 100
query_cache_limit = 64M // 32M
query_cache_size = 64M // 32M
query_cache_type = 1 // 1
// + query_cache_min_res_unit=32kb
tmp_table_size = 256M // 32M
max_heap_table_size = 256M // 32M
// + key_cache_division_limit=40
// + low_priority_updates=1
// + thread_concurrency=8
// + concurrent_insert=2


log-slow-queries = /var/log/mysql-slow.log
long_query_time = 2
(I'm guessing that's a BIG log. Seems very low. Maybe 10-20?).

P.S. Not running any innodb. I've read conflicting advice, but the biggest drawback is 2-3X disk space.

P.S.S. Are you running Sphinx search? On boards with over 1M posts fulltext search can bring any server to its knees.

P.S.S.S... :P One more thing to try. Add to your robots.txt
User-agent: *
Crawl-Delay: 1

Link to comment
Share on other sites

Thanks for the info.

It definitely helps seeing other people's set-ups - my main concern is that I don't allocate too much memory to area's that don't need it or where it won't be of benefit.

I've taken your advice on the crawl delay and added that in - I'm already using Sphinx for search so search speed isn't too much of an issue - and funnily enough I don't have a mysql-slow.log file anywhere, never mind a big one.

AFAIK the only table I have as innodb is the sessions table, but disk space isn't really an issue for me so if there is an advantage to be had by converting I'd be interested to know how to do so?

I'll carry on tweaking MySQL to see what improvements I can make, but appreciate you taking the time to assist.

Cheers,
Shaun :D

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.

×
×
  • Create New...