hugely increased load after upgrade to v.31

[Cosmic]

I've seen a huge extra load on my server after upgrading from v2.36 to v.3.1.

It's gone from running at around a load of 1, to running at a load of 5 or greater (without anything else having changed).

Is that normal? Or is there something else I should be looking at as the sudden cause of this?

[Biker.GA]

What are your top running processes? You should be able to determine this by going to Support in your ACP and looking at your process list.

[Robulosity2]

I've seen a huge extra load on my server after upgrading from v2.36 to v.3.1.

It's gone from running at around a load of 1, to running at a load of 5 or greater (without anything else having changed).

Is that normal? Or is there something else I should be looking at as the sudden cause of this?

There is more behind 3.x than there was in 2.3.6 so seeing a load increase when doing that version change isn't unusual.

[3DKiwi]

How much memory has your server got? I had to upgrade my server from 1gb memory to 2gb of memory after upgrading to version 3. With 1gb it would crash regularly.

3DKiwi

[Cosmic]

There is more behind 3.x than there was in 2.3.6 so seeing a load increase when doing that version change isn't unusual.

It's far more than just a little more overhead - it's a HUGE amount more.

[Cosmic]

How much memory has your server got? I had to upgrade my server from 1gb memory to 2gb of memory after upgrading to version 3. With 1gb it would crash regularly.

3DKiwi

dedicated server, 8GB RAM, xeons, etc, etc, etc. Not a baby server by any means - it's a server that kept running OK with 3,000+ very active users with version 2.36, but which is now struggling to cope with 800 not very active users.

At this moment I very much wish I'd never touched v3.1. It's a dog in my experience, and the only thing I can find to have it run reasonably OK is to use the 'performance mode' - which turns off all the features which led me to upgrade in the first place.

My tip: avoid like the plague.

[Bono]

I upgraded also from 2.3.6 to 3.0.5 and I didn't experienced any performance hit. Load is slightly higher but nothing to worry about, but then again my server is highly optimized, also xeon with 4gb of ram and LiteSpeed web server.

Post some info so we can see whats wrong:
like top, free -m.

[Cosmic]

too busy trying to get the new v311 release in place in the hope that some of the issues are resolved in that to mess around with screen dumps, soz.

[Mikorist]

I just implemented eAccelerator .... Zend Optimizer made php just a hair faster, but eAccelerator is FAST AS HELL on IPB 3.1.x
Pages come up like 30% more faster and produce less load on system ...

Lighttpd + Zend + eAccelerator + Good Server = The Win

[Cosmic]

I just implemented eAccelerator .... Zend Optimizer made php just a hair faster, but eAccelerator is FAST AS HELL on IPB 3.1.x

Pages come up like 30% more faster and produce less load on system ...

Lighttpd + Zend + eAccelerator + Good Server = The Win

To speed things up I've:-

- installed Sphinx
- changed the sessions table to HEAP
- installed XCache

It's certainly an improvement, but is still significantly slower than v2.36 was without those things.

[MarcusInMd]

What Mysql version are you running? 3.x crushed our dual server setup that was humming along with 2.3.6

[ikillbill]

We had same issues, IPB 3 is using MUCH more server resources
We did complain this on 3.0.5, unfortunately issue is not resovled until now

[Cosmic]

What Mysql version are you running? 3.x crushed our dual server setup that was humming along with 2.3.6

I'm running v5.0.77 - so it's not anything related to v3.x

[MarcusInMd]

I would beg to differ though. I do believe that 3.x is significantly more taxing on mysql.

I am finishing up my database server upgrades and will be redeploying in the next few weeks. Once it's in place I am converting the posts table over the innodb and hopefully never have to look back.

since upgrading to 3.1.1 I am noticing that deleting posts/topics is taking even more server load than before. Someone did a delete earlier in the day and our server spiked up to 8 from around 1.




That is just nuts. Locks out everything until the process completes too.

[daks001]

I can say that I also saw a large increase in server load especially going from 3.0.5 to 3.1.1 (and yes there was also a large increase going from 2.3.6 to 3.1) in fact after a day or two, 3.1.1 came to a screeching halt on my server!
Doing a SHOW PROCESSLIST either in SQL or in the Support Section of ACP and I'd see

Locked SELECT * FROM ibf_core_item_markers WHERE item_member_id=38326 AND item_is_deleted=0

Rooting around on the forums here I came upon this thread;

http://community.invisionpower.com/topic/315060-forums-slowed-to-a-crawl/page__p__1980944__hl__ibf_core_item_markers__fromsearch__1#entry1980944

Ah Ha bad sql "optimization" so I backed up and applied the (no ibf_core_item_markers_dev in my db)

ALTER TABLE `ibf_core_item_markers` DROP INDEX `item_member_id` ,

ADD INDEX `item_member_id` ( `item_is_deleted` , `item_member_id` )

command into the SQL box in phpmyadmin and YAY the server can keep up.... is it anywhere close to the speed of 2.3.6 NO!

This is fixed in 3.1.2 :)

I just hope... the developers put some of the new features on the backburner and spend a little bit more time working on optimizing the way the dbase/system works... No matter how "shiny" a forum is, if it moves like a slug it's no darn good to anybody! Nor does everyone want to use IPS hosting, or feel you should need a 4 core cpu with multi-disk array SAS drives and 12gig of ram just to run a forum for 60k users... Sorry but the forum software is heading that way.


I'm wondering if changing the ibf_core_item_markers to an innodb will help, or if there are any other areas that would also benefit from switching to innodb. I'm still rooting around the forums to see if there is any recomendations from the IPS masters. :)

[Connor T]

Things become more robust, and thus require more resources. Not many people run Windows XP on Commodore 64's.

Point being. I've seen many sites with 3000+ users online at once and they are running fine. It just comes down to optimizing your server, sometimes separating mysql and files, using a caching method, enabling sphinx, and not allowing file uploads. (that murders servers)

[SecondSight]

Hello ! :)

Just because you see web pages loading more or less fast doesn't mean there are no problems. :) For instance, my board looks like it is doing well since web pages load fine, but in fact there are high server loads at moments (from about 1.8 to about 14, but my server is powerful enougth to face it).

[MarcusInMd]

Things become more robust, and thus require more resources. Not many people run Windows XP on Commodore 64's.

Point being. I've seen many sites with 3000+ users online at once and they are running fine. It just comes down to optimizing your server, sometimes separating mysql and files, using a caching method, enabling sphinx, and not allowing file uploads. (that murders servers)

I disagree with these comments. Our two servers are very optimized, have loads of resources at their disposal and since 3.x and now even more so with 3.1.1 we have server issues. There are some instances where this software is causing issues and I have a very hard time believing that it's entirely a server optimization issue.

[Painted Horse]

Hello ! :)

Just because you see web pages loading more or less fast doesn't mean there are no problems. :) For instance, my board looks like it is doing well since web pages load fine, but in fact there are high server loads at moments (from about 1.8 to about 14, but my server is powerful enougth to face it).

Curious, What is your server's hardware spec?

[Bono]

Post your traffic stats like unique visitors, page views and config.

Almost everybody say their server is fully optimized but most of servers I looked at was just badly optimized. Most of the time they did just copy pasted mysql/apache config files.

Like someone said before:

Switch from apache to some lightweight web server, if you are in shared environment then use nginx and if shared use LiteSpeed.

I haven't noticed any noticeable increase in load when switching from 2.3.6 to 3.0.5, and i'm running single quad core x3220 with 4GB of ram on SAS disks. My avg. load is 1 or 0.25% and i'm using 25% of memory. My forum is not that big only 150k posts, around 2k unique and 30k page views. But my site has which is on same server 50k unique and 550k page views per day.

MYSQL QPS is now at 230 at peak times i have few K QPS.

[Cosmic]

Almost everybody say their server is fully optimized but most of servers I looked at was just badly optimized.

Well, I've had the IPB folks look at my setup, and there was very little they recommended to change.

I made those changes and accept that they have improved things slightly, tho those improvements are not visibly noticeable, but the far higher load with v3.1 very much is.

I'm left with the situation where I either spend out a VERY significant amount on a much higher spec server (while already having a high spec server) to cover the needs of the various traffic peaks I get on my seasonal-traffic-ed website, or I give up caring that my forum users will find the forums unusable at too many times thru the year.

Given that the bankers have robbed us all, I really only have one of those options available. And unfortunately that's then a poor advert for IPB.

[MarcusInMd]

Post your traffic stats like unique visitors, page views and config.

Almost everybody say their server is fully optimized but most of servers I looked at was just badly optimized. Most of the time they did just copy pasted mysql/apache config files.

Like someone said before:

Switch from apache to some lightweight web server, if you are in shared environment then use nginx and if shared use LiteSpeed.

I haven't noticed any noticeable increase in load when switching from 2.3.6 to 3.0.5, and i'm running single quad core x3220 with 4GB of ram on SAS disks. My avg. load is 1 or 0.25% and i'm using 25% of memory. My forum is not that big only 150k posts, around 2k unique and 30k page views. But my site has which is on same server 50k unique and 550k page views per day.

MYSQL QPS is now at 230 at peak times i have few K QPS.

Total Hits 	109578431

Total Files 	29301959

Total Pages 	9953626

Total Visits 	934964

Total KBytes 	543802030

Total Unique Sites 	81812

Total Unique URLs 	49370

Total Unique Referrers 	4859

Total Unique User Agents 	11171

. 	Avg 	Max

Hits per Hour 	147282 	1759113

Hits per Day 	3534788 	6264126

Files per Day 	945224 	1562829

Pages per Day 	321084 	607763

Visits per Day 	30160 	49438

KBytes per Day 	17542001 	27626706

That is what our traffic looked like last December. On the 18th I pulled our DB server offline and have not used it since. I can tell you flat out that moving the DB off our DB server which had a 4.x version of Mysql made a huge difference in performance but still we had issues. The web server which is where our DB is presently has Mysql 5.x on it.

[SecondSight]

@ dancingbear :

All performance problems were resolved for me after I had a query problem fixed : http://community.inv...e-item-markers/ You can look and see if you have the same problem.
I also had flood (DOS ?) problems and I had an Apache module called mod_evasive and some antidos software installed on my server (this was done last wednesday).

[Cosmic]

thanks - I'll raise a ticket and see if I can get them to do the same as that thread doesn't actually say what they've done - I've got a mega-peak coming up on Sunday, so need everything running as quickly as it's possible to be.