Slow forum and topic page load times

[Tabs]

We've migrated our biggest site to IPB but haven't switched over yet as we're still testing.

I've noted very slow page load times when viewing Forums and Topics. Takes roughly between 4-6 seconds after clicking a post for it begin loading and then a further 2-4 seconds to complete. Forum views are slightly quicker but still subject to a delay. Other pages like Classifieds load instantly.

As this is a test site we only have 2-4 users online so we're not experiencing heavy traffic conditions.

What can we check / configure to speed things up?

[Dmacleo]

I wonder if not live and not sending notifications then having too many "followed" items bogging it as it can't send.

[Tabs]

We have no notifications set for users, the migration script didn't copy these user settings over.

What can we use to see what's causing the slow of forum and topics? Are there any query analysis tools out there to see where the bottleneck is?

[xbp-tpn]

Well you can use something like gtmetrix.com or webpagetest.org to check which part of IPS installation is taking the longest to load, however this would only work if your test site is online and points to a valid domain name.

[Tabs]

Test site is live on a domain but we've got IPB site locked to prevent access. I'll try opening up temporarily and running those tools.

[Dmacleo]

no tapatalk or forum runner app (or any app) installed that does push notifications to members is there?

[Tabs]

No we haven't installed either of those plugins.

Would our DB table type choice have an impact on load times? We're using myisam and have around 2.6 million posts in our posts table. Wondering if this is causing an issue.

[Tabs]

Ran an analysis with gtmetrix, there is one module that stands out a taking between 1 and 3 seconds to load when a topic is loaded:

http://domain.com/index.php?s=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx&app=core&module=task

This is the search component of the topic call?

[wimg]

Actually, a site with 2.6 million posts does take fairly long to load if it isn't being run on a biggish server and not optimized too well.

I couldn't tell you what the task core module exactly does, unfortunately - maybe someone else can help with that.

HTH, warm regards, Wim

[Dmacleo]

saw the task and was wondering about ping services (which will cause issues on test site I think) but that should only apply when actually posting a new topic not opening one.

[xbp-tpn]

That file takes a lot of time to load at my end as well, always wondered what it did.

[Tabs]

Actually, a site with 2.6 million posts does take fairly long to load if it isn't being run on a biggish server and not optimized too well.

I couldn't tell you what the task core module exactly does, unfortunately - maybe someone else can help with that.

HTH, warm regards, Wim

The server is fairly well powered and both IIS and MySQL configured with ample resources available.

Would changing storage engine to InnoDB improve performance?

[wimg]

The server is fairly well powered and both IIS and MySQL configured with ample resources available.

Would changing storage engine to InnoDB improve performance?

It might indeed, although you may want to consider an alternative mySQL engine like Percona or MariaDB as well in that case. From what I remember reading, they have some additional performance enhancements with InnoDB.

I see you are using IIS - we use Apache and NGINX, and that seems to do the trick more or less, provided we don't have more simultaneous sessions than about 6000-7000, on a server with 8GB of RAM and 8 cores on CentOS. We are at the limit, however, it appears, possibly because of, amongs others, the amounts of email the server has to process as well. It is not entirely clear anymore in our case either what the bottleneck is currently. We are running a VM box on our dedicated server, however, which appears to make finding exactly where the bottleneck is a bit more difficult.

BTW, we have 2.6 million+ posts as well.

HTH, warm regards, Wim

[Tabs]

Hey Wim thanks for the reply.

We're on a dedicated Server with 16GB RAM and 16 cores. It is running our original site at the moment, which is IIS and MSSQL based and consumes a helluva' lot of resource! So there may be some resource bottlenecks skewing our view like you mention. What I'll do is organise some downtime on the main site (taking MSSQL offline), so we're in essence running cold to do some stress testing on our IPB installation.

We're constrained by the Server OS (Windows 2008 x64) - only MariaDB is an option from those two you mention. I'll take a look at that as an alternative to MySQL.

Our other option is to Archive some of the older posts but I'm a stiffler for old topics being dug up and stimulating new activity!

[wimg]

We like old topics to be re-awakened as well. On our board older topics may deal with very similar questions and discussions as newer ones, so if someone adds to an older topic, so much the better :).

I have no experience with MSSQL and/or IIS, I must say, but I always got the impression they require more resources than mySQL and Apache do - I may be entirely mistaken, however. I just inherited what we had, and moved on from there (started with IPB 1.30 if I am not entirely mistaken :D).

Warm regards, Wim

[Tabs]

IIS does work differently to Apache due to it being centered around .NET application framework etc. Both platforms have their advantages - we're using it as it's more cost effective to stick with the current setup whilst we migrate. If it fairs well we'll keep with it, otherwise Linux all the way.

Question to other large IPB forum admins with over 2 million posts - what table types are you using?

Edit to add - After reading this post and adding $INFO['use_wincache'] = '1'; to conf_global.php we seem to fetching topics a little quicker. However using the search system is painfully slow at 10+ seconds average for any query.

[Jinkler]

myisam here.

[Jinkler]

However using the search system is painfully slow at 10+ seconds average for any query.

There is an add-on in the marketplace to offload your search to google, "CSE-Google" I think it's called.

[xbp-tpn]

There is an add-on in the marketplace to offload your search to google, "CSE-Google" I think it's called.

But wouldn't that only work effectively if all your pages in IPB are indexed by google? If you board's SEO isn't that great or if some of your pages are not indexed by google (restrictions on guest viewing e.g.) then your members wont be able to use the search function effectively.

[Tabs]

I'll try setting up Sphinx and testing that to see what improvements it makes to search response times. Would prefer to keep our search index in house rather than farm out to Google - wary of placing too many eggs in their basket!

[Dmacleo]

least with sphinx you can actually search.

just tried to search for a hook with word color in the title and market would not allow me grrr.

[Tabs]

Update: Ran some intensive tests with around 10-15 users and our main site offline. Each user ran multiple tabs with auto refresh, hitting new topics repeatedly for a period of 15 mins. Performance was great, only noticeable delay came from opening topics or running New Content for the first time - after which the data was then cached and became instantly available. There were a couple of peaks in CPU activity but these were extremely short in duration. These look to be cased by the php engine firing up an individual thread for each requesting coming into the site - which was expected due to the way IIS php works.

I've now also setup Sphinx which has reduced search times - again this seems to be affected by queries run for the first time taking slightly longer than repeat queries.

We're now in a position to go live.... Wish us luck!