Poor Performance - High load

[DaveLegg]

We migrated our rather sizeable board to 3.0.5 yesterday, and as soon as we launched the load averages on our apache servers went through the roof (>30). We submitted a critical ticket, and are yet to hear back. Even with the board set to "Offline", the load on our front end servers is around 10 on each of them, with IPB 2 running normally, we would rarely hit a load of 1. Renaming index.php to something else to prevent access to the forums results in the load dropping back to normal levels, so we know the forums are the cause of the problem. Enabling "High Performance Mode" had no effect on the load, and setting the forums offline is still not enough to resolve the problems.

Does anyone have any suggestions of what we can try?

[Mat Barrie]

You won't get any answers in community feedback. Just log into the forums with the same email and password you bought the software with, and post in peer to peer support. What people will probably need to know to help you will be:

MySQL version
Web server you're using (Linux/Apache, Linux/Lighttpd, Windows/IIS)
PHP version (and whether it's CGI, SAPI, or FastCGI)
Specs on server (dedicated, right? Virtual or Physical?)

You also said Apache servers. It might help to know what clustering solution you're using. Is it real clustering, load balanced with Netscalers or F5s, DNS balanced, etc.

[Zhana]

Facing the same issue from Friday.

[Twisted Gamer]

Facing the same issue from Friday.

If your the same person as OP, then you'll need to start with what was recommended in the post above yours or start your own thread as not all setups are the same.

The more info you can provide (even if you don't think its relevant might give others some insight).

With my upgrade the only thing that cause my load to spike like the OP was attempting to edit an existing download in IP.Downloads that didn't migrate correctly.

Restarting Apache brought the load back down to normal. I currently have a ticket in progress with IPS about it. Maybe my issue is relevant to the O.P.

[wimg]

That is what we saw happening too, over the last few weeks, since the upgrade to 3.0.5.

Essentially we undertook a major effort at tuning our setup, and it is running fine now, apart from some weird problems some members have, but occasionally Apache starts going wild, and the only thing you can do in that case is force a reboot and keep your fingers crossed. We had cpu-cpount go up to 50.71 at some stage, before we could reboot. And we only have 4 cpus (quad-core).

We are contemplating moving to a bigger server now. Under 2.3 we were running kinda ok, but 3.x is asking many more resources, it seems.

Kind regards, Wim

[MarcusInMd]

Get used to it. The problems are likely Mysql related. We have a very active forum and since upgrading to 3.0.x we have had nothing but issues. Our servers are top notch too with 16gb and 12gb and both are dual quad core servers.

I appreciate the help that IPB (particularly Brandon) but nothing they been able to provide to help has done anything.

I can tell you that if you are using Mysql 4.x for your database engine you are likely going to have even more issues. We had severe table locking going on with our DB server which is using Mysql 4.1x and had severe load issues around 600 to 700 users online at once..We managed over 1300 without table locking in IPB 2.x with this same server. I moved our database over to our web server which is using Mysql 5.0x and it helped but we recently started to see major table locking problems once again when we hit a peak of 1650 to 1750 users on at once. This only happens during heavy influx of traffic. When things are kinda of idle and those 1700 are not doing much all is well. But when many start posting or reading the same threads things go south in a hurry.

[3DKiwi]

Have you added a delay to your robots.txt file? Plus configured your Google Webmaster account to slow down Google indexing your site.

My site plus the Neowin site has issues after upgrading to version 3. I had to upgrade my server memory but just editing the robots.txt file to slow down the search engines made a significant difference. My guess is search engines index pages much quicker with version 3 than they did with version 2.

3DKiwi

[MarcusInMd]

Have you added a delay to your robots.txt file? Plus configured your Google Webmaster account to slow down Google indexing your site.

My site plus the Neowin site has issues after upgrading to version 3. I had to upgrade my server memory but just editing the robots.txt file to slow down the search engines made a significant difference. My guess is search engines index pages much quicker with version 3 than they did with version 2.

3DKiwi

Care to explain? Perhaps a link with more info on this? Google is always on our site.

[3DKiwi]

Delay in robots.txt explained here.

Google crawl rate explained here.

3DKiwi

[MarcusInMd]

Delay in robots.txt explained [url="http://drupal.org/node/14177"]here.[/url]

3DKiwi

Thank you. I already setup the robots.txt file and applied it to our httpdocs folder. In the mean time I turned bot logging back on and I noticed that google really is not doing much. In the past 15 minutes it has logged 384 hits to the site. Seems like that is not enough to cause any damage. Also bing and Ask Geeves have been on but barely doing anything. As for the google crawl rate. I don't have an accoutn setup with google so I don't think this will be useful.

This is my robots.txt file I am using now. I understand it takes a few hours to kick in.

User-agent: *

Crawl-Delay: 10

Disallow: /bb/admin/

Disallow: /bb/cache/

Disallow: /bb/converge_local/

Disallow: /bb/hooks/

Disallow: /bb/ips_kernel/

Disallow: /bb/retail/

Disallow: /bb/public/js/

Disallow: /bb/public/style_captcha/

Disallow: /bb/public/style_css/

Disallow: /*app=core&module=usercp

Disallow: /*page__st*$

Disallow: /*page__f__*$

Disallow: /*page__tab__*$

Disallow: /*page__view__*$

Disallow: /*view__getnewpost$

Disallow: /*view__getlastpost$

Disallow: /*view__old$

Disallow: /*view__new$

Disallow: /*view__getfirst$

Disallow: /*view__getprevious$

Disallow: /*view__getnext$

Disallow: /*view__getlast$

Disallow: /calendar/

[MarcusInMd]

Is anyone elses Agent User Group list empty before configuring it? I have no user agents listed that I can add to a new group.

[3DKiwi]

As far as I know the robots.txt file should be in your root directory i.e. public_html on a dedicated server.

3DKiwi

[SecondSight]

Hello ! :)

Has someone found a solution to this problem ?

Thank you ! :)

[Fmoraes]

Try disabling some functions of Ipb 3.0.5, it helps if you have little access to robots

Enable the search engine spider recognition? No
Log all spider visits? No
Show spider/bot in the active users list? No