Server Loads on IPB 3.x versus 2.3.x

[Fast Lane!]

I will update this thread as time goes on, but my premise is simple. IPB 3.x has much higher loads than 2.3.x. Here is yesterdays loads. Can you tell when the new forums went live with new traffic/loading? For comparison I am putting the previous weeks (same day) loads. I am not griping on slow queries (don't see any) but due to all the ajax (or whatever) there is some serious CPU burden on my server.

[Connor T]

These threads are getting annoying. I'm sorry I'm unloading it on you. Things upgrade. Harddrives have to be bigger since programs are larger. Same to be said with webservers. More robust features are added, thus more memory is needed. The new default in php5 is 128M compared to a pathetic 64M that was in php4 (i think)

Point being, yes it uses more. It also is quite different than 2.3.x requiring a different memory load.

[3DKiwi]

A few of us on medium to large sites experienced similar things. Things that you can do:

1. Upgrade server memory. 2.3.6 ran fine on my dedicated server with 1gb of memory. with version 3 I was getting frequent crashes. The cure was to upgrade to 2gb of memory. I have not had a crash since.

2. Add a delay to robots.txt to slow down search engines. Seems search engines go bananas indexing your forum pages with version 3. I added a 10 second delay to mine. e.g. here on line 2:

User-agent: *

Crawl-Delay: 10

Disallow: /ipb/admin/

Disallow: /ipb/cache/

3. Slow down Google indexing site. Done in your Google Webmaster account. if you don't have one then you will need to create one.

4. Make sure you running the latest version of IPB i.e. 3.0.5

Hope this helps.

3DKiwi

[Fast Lane!]

These threads are getting annoying. I'm sorry I'm unloading it on you. Things upgrade. Harddrives have to be bigger since programs are larger. Same to be said with webservers. More robust features are added, thus more memory is needed. The new default in php5 is 128M compared to a pathetic 64M that was in php4 (i think)

Point being, yes it uses more. It also is quite different than 2.3.x requiring a different memory load.

Umm... This is my server primarily only for the site I just upgraded:

Dual Quad Core Intel Xeon L5420 @ 2.50GHz
8 GB 667 MHz DDR2 Dual Rank, Fully Buffered RAM
15K RPM SCSI/SAS Hard Drives in RAID 10
Centos 5.x
php 5.2.13, Mysql 5.0.89, apache 2.2.15

I am not on a wimpy server. I have eaccelerator using an extended cache of 96MB (defaults to 32MB for most people). I have massive caches and well tuned my.cnf and apache.

@3DKiwi. Thanks for taking the time to post :). I had tried the crawl delay but only set to 1 second (per neowins similar decision). Did you try this time and have trouble still? My robots.txt is as follows:

User-agent: *

Crawl-Delay: 1

Disallow: /forums/admin/

Disallow: /forums/cache/

Disallow: /forums/converge_local/

Disallow: /forums/hooks/

Disallow: /forums/ips_kernel/

Disallow: /forums/retail/

Disallow: /forums/public/js/

Disallow: /forums/public/style_captcha/

Disallow: /forums/public/style_css/

I did not throttle google yet. That was the next step although I always hate to limit them in any way.

[Fast Lane!]

Here is the main problem. Look at top and see how many apache threads are sucking up CPU power (small but significant per process)... Notice all 8 XEON cpus having to work hard to keep my server afloat.

top - 20:01:27 up 19:52, 2 users, load average: 22.43, 18.54, 14.55

Tasks: 151 total, 12 running, 139 sleeping, 0 stopped, 0 zombie

Cpu0 : 69.5%us, 6.8%sy, 0.0%ni, 23.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu1 : 57.1%us, 21.4%sy, 0.0%ni, 21.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu2 : 75.0%us, 6.7%sy, 0.0%ni, 18.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu3 : 59.6%us, 10.5%sy, 0.0%ni, 29.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu4 : 57.9%us, 5.3%sy, 0.0%ni, 36.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu5 : 72.7%us, 20.0%sy, 0.0%ni, 7.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu6 : 68.5%us, 13.0%sy, 0.0%ni, 18.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Cpu7 : 77.1%us, 12.5%sy, 0.0%ni, 10.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Mem: 8388608k total, 1855156k used, 6533452k free, 0k buffers

Swap: 0k total, 0k used, 0k free, 0k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

19818 mysql 16 0 563m 263m 4604 R 26.4 3.2 50:43.51 mysqld

21772 nobody 16 0 127m 26m 14m R 8.0 0.3 0:02.00 httpd

28342 nobody 15 0 126m 18m 6928 S 6.8 0.2 0:00.23 httpd

21507 nobody 16 0 126m 23m 11m R 6.2 0.3 0:01.22 httpd

27684 nobody 15 0 126m 20m 9384 R 6.2 0.3 0:00.32 httpd

28341 nobody 15 0 126m 18m 7212 S 5.6 0.2 0:00.19 httpd

27683 nobody 15 0 126m 20m 9024 S 4.8 0.2 0:00.59 httpd

17409 nobody 15 0 126m 28m 16m S 4.5 0.3 0:04.03 httpd

28338 nobody 15 0 125m 18m 7176 S 4.5 0.2 0:00.21 httpd

15445 nobody 15 0 124m 34m 24m R 4.2 0.4 0:03.76 httpd

23855 nobody 16 0 126m 23m 12m S 3.3 0.3 0:02.01 httpd

9709 nobody 15 0 125m 27m 16m S 2.1 0.3 0:04.43 httpd

21660 nobody 15 0 126m 24m 12m S 2.1 0.3 0:01.87 httpd

27674 nobody 15 0 126m 22m 10m S 1.8 0.3 0:00.64 httpd

27667 nobody 16 0 125m 19m 9028 R 1.5 0.2 0:00.47 httpd

5258 nobody 15 0 125m 25m 14m S 1.2 0.3 0:05.03 httpd

23853 nobody 15 0 126m 24m 12m R 1.2 0.3 0:01.88 httpd

23871 nobody 16 0 126m 21m 10m S 1.2 0.3 0:00.52 httpd

17898 nobody 15 0 124m 23m 14m R 0.9 0.3 0:03.06 httpd

19875 nobody 15 0 124m 22m 13m S 0.9 0.3 0:02.30 httpd

20102 nobody 16 0 126m 27m 15m S 0.9 0.3 0:01.60 httpd

13572 nobody 15 0 125m 22m 12m S 0.6 0.3 0:03.25 httpd

21508 nobody 16 0 126m 24m 12m S 0.6 0.3 0:01.79 httpd

24049 nobody 15 0 126m 21m 10m S 0.6 0.3 0:00.82 httpd

26275 nobody 15 0 126m 20m 9368 R 0.6 0.2 0:00.25 httpd

26623 nobody 15 0 125m 20m 9576 S 0.6 0.2 0:00.80 httpd

9673 nobody 16 0 126m 29m 18m S 0.3 0.4 0:05.66 httpd

19805 root 18 0 122m 11m 3904 S 0.3 0.1 0:13.54 httpd

19876 nobody 15 0 125m 22m 11m S 0.3 0.3 0:01.71 httpd

20308 nobody 16 0 126m 22m 11m S 0.3 0.3 0:01.34 httpd

21764 nobody 15 0 126m 22m 11m S 0.3 0.3 0:01.12 httpd

23857 nobody 16 0 125m 22m 11m S 0.3 0.3 0:00.88 httpd

23879 nobody 16 0 125m 21m 10m S 0.3 0.3 0:01.78 httpd

24047 nobody 15 0 126m 23m 11m S 0.3 0.3 0:01.12 httpd

27659 nobody 16 0 125m 19m 9136 S 0.3 0.2 0:00.67 httpd

27660 nobody 15 0 124m 18m 8548 S 0.3 0.2 0:00.17 httpd

27661 nobody 15 0 126m 23m 12m S 0.3 0.3 0:00.38 httpd

27671 nobody 15 0 124m 19m 9.8m S 0.3 0.2 0:00.41 httpd

28340 nobody 16 0 122m 12m 4836 S 0.3 0.2 0:00.01 httpd

[Mark]

If your load is of an unacceptable level rather than just a couple tenths higher (which it looks like from that screenshot) I would recommend submitting a ticket so a technician can take a look at your site.

[3DKiwi]

Hi. No I haven't tried 1 second but my server would probably cope with 1 second. What I also did is block a few bad search engines either in robots.txt or by blocking IP's via cPanel. Google I allow but slowed them down a bit.

Have you files a support ticket with IPS? They may be able to offer some suggestions.

3DKiwi

[Fast Lane!]

I filed one today. THey have not logged into my ACP or started working it yet -- I hope today as tomorrow I am very busy.

If your load is of an unacceptable level rather than just a couple tenths higher (which it looks like from that screenshot) I would recommend submitting a ticket so a technician can take a look at your site.

Did. Waiting (desperately) for help :). Right now a load of 17 at 9:30PM PST. At this hour of day that is amazing.

[bfarber]

For what it's worth, there's a 5-10 page topic in the customer lounge where several customers found that eAccelerator was causing their server to crash for whatever reason. Uninstalling eAccelerator resolved the high loads/crashing (I believe most of them ended up installing xcache afterwards).

There are a lot of things we can look into for you. You can't tell from a graph where the problem is. To look into this for you, however, we'll need you to submit a ticket.

[Fast Lane!]

Yes the old version had a problem with cpanel. That error was fixed and I have the revised version of both pieces of software that play well together. I had to go through that in January (was another fun adventure in finding out what was wrong). I would like to hope that was it but because I was indeed affected by it previously (and now fixed) I eliminated that as a potential cause.

[bfarber]

That's the sort of thing we can better diagnose through a ticket, since we'd have direct access to your server. :)

As you already have a ticket in, our technicians will be able to look more closely at the issues you are facing during normal business hours.

[Fast Lane!]

Agree. I have had senior technicians at my NOC look at the server and the configuration on that end. All is good there. No known issues nor software changes occurred other than upgrading the forums.

[Zone Plate]

I am seeing a 4-5x increase in page execution times (0.03 vs. 0.15) between a massive 2M post 2.3.x install (the faster one) vs. a 200k post 3.1.2 install (the slower one)...

Unless I can get that execution time down, I don't think I can upgrade my big board. :(

[bfarber]

You should submit a ticket if you are facing resource issues. There are a huge varying number of things that can affect performance, and there's no magic "turn this setting on" that we can tell you from here without having proper access.

(Note, also, that this topic is 3-4 months old too)