Invision Community 4: SEO, prepare for v5 and dormant account notifications By Matt Yesterday at 02:04 PM
wegorz23 Posted May 7, 2019 Posted May 7, 2019 Hi, we have problem with our site - www.audiostereo.pl. Site running not so fast, but acceptable, but adding posts its lottery. Sometimes its 2-5 seconds, sometimes even 10 and more. It was even 30-60 seconds, but Ive changed search engine rules. Now it takes data from only last 3 years. Search index was 4gb+ so that was one of the reasons. After cutting serach it a little bit better now - but still to slow for our users. We have 2 servers - one with apache for files and 2nd one for db. We are using MariaDB and its mixed - https://ibb.co/fkRGzbq some tables on InnoDB, some on MyISAM. Our servers are not so old I think - one for db is from 2 months ago, second one for files is from begging of 2018, so I believe they are good enough for site like this. Its rather some problem with configuration or sth. We have newest version of IPS installed. Any help please?
Gabriel Torres Posted May 7, 2019 Posted May 7, 2019 Some tips: 1. Upgrade all MySQL tables to InnoDB 2. Install and configure Redis 3. Make sure PHP is using opcache 4. Use a CDN like Cloudflare
wegorz23 Posted May 7, 2019 Author Posted May 7, 2019 Ok, thank you for this tips. We will make it and then back to this topic 🙂
wegorz23 Posted May 7, 2019 Author Posted May 7, 2019 Ok, I have been installed this Cloudflare by myself, and have changed DNS names. We will see after 24h is it works. We have free plan. We need to make some more in cloudflare config?
Stuart Silvester Posted May 7, 2019 Posted May 7, 2019 More often than not, when we see reports of slow content submissions, it's caused by a slow (or non-working) email setup (typically, the SMTP credentials are no longer working, or the SMTP service is slow to respond). If this is the case, you may see something in the email logs (AdminCP > System > Emails > Error Logs)
bfarber Posted May 7, 2019 Posted May 7, 2019 I second #1 as well - make sure you get all tables over to InnoDB (which supports row-level locking for most query types, rather than table-level locking), and then make sure you optimize your MySQL configuration for InnoDB (if it is optimized for MyISAM then simply switching table types may not be too great on its own). It is possible for instance that one user starts searching, which locks your search table, and then another user is posting - that post may have to wait for the search to finish before the search index insertion can execute (just a random example).
Gabriel Torres Posted May 7, 2019 Posted May 7, 2019 10 hours ago, wegorz23 said: Ok, I have been installed this Cloudflare by myself, and have changed DNS names. We will see after 24h is it works. We have free plan. We need to make some more in cloudflare config? 1. Make sure that in the DNS panel, your domain is with the orange cloud activated, meaning cloudflare is active. From your PC, do a nslookup <yourdomainname> and you should get cloudflare's IP address instead of your server's IP address. 2. Speed tab: I have Autominify for HTML only (in the past minimizing JS and CSS gave us some trouble, but I ought to re-test this sometime), Polish = lossy + webp, Enable Accelerated Mobile Links, enable Brotli, enable Mirage, Rocket Loader: I have this disabled because in the past it gave us some trouble, but I ought to re-test this sometime. 3. Caching tab: Caching level: standard 4. Fireall > Settings (this is at the upper right corner of the page, easy to miss it), Security Level: High. This was a key configuration for us here. After we enabled this to "high", performance improved a lot. I guess we were being bombarded with rogue spiders/crawlers. Also make sure Browser Integrity Check is enabled. I hope this helps for CloudFlare.
sudo Posted May 8, 2019 Posted May 8, 2019 11 hours ago, Stuart Silvester said: More often than not, when we see reports of slow content submissions, it's caused by a slow (or non-working) email setup (typically, the SMTP credentials are no longer working, or the SMTP service is slow to respond). If this is the case, you may see something in the email logs (AdminCP > System > Emails > Error Logs) Are email notifications not queued to be seperate from the post submission process or do they just stream out when someone submits? We just have some very large topics with a lot of people that would be mailed and this could be a bit of an issue as right now they are using queuing on vbulletin.
wegorz23 Posted May 8, 2019 Author Posted May 8, 2019 9 hours ago, Gabriel Torres said: 1. Make sure that in the DNS panel, your domain is with the orange cloud activated, meaning cloudflare is active. From your PC, do a nslookup <yourdomainname> and you should get cloudflare's IP address instead of your server's IP address. We have it like that: Nslookup shows me our sever IP, so I think that propagation of DNS is not ready yet. We must wait. Reset set as you said. We have free plan for test now. We will change to better one maybe in future after some tests.
wegorz23 Posted May 8, 2019 Author Posted May 8, 2019 19 hours ago, Stuart Silvester said: More often than not, when we see reports of slow content submissions, it's caused by a slow (or non-working) email setup (typically, the SMTP credentials are no longer working, or the SMTP service is slow to respond). If this is the case, you may see something in the email logs (AdminCP > System > Emails > Error Logs) We have one error that repeats all the time:
Stuart Silvester Posted May 8, 2019 Posted May 8, 2019 9 hours ago, sudo said: Are email notifications not queued to be seperate from the post submission process or do they just stream out when someone submits? We just have some very large topics with a lot of people that would be mailed and this could be a bit of an issue as right now they are using queuing on vbulletin. It will send them immediately if there are a small number of emails to send. If there are many, they will be sent in the background after content submission. 1 hour ago, wegorz23 said: We have one error that repeats all the time: This particular error message means that you're using PHP mail to send a message, but it's failing and isn't returning an error message (typically it would). You may want to try temporarily switching to a SendGrid account (they have a free trial) to see if that speeds up the submission process, if it does the issue is with your PHP mail/server configuration
wegorz23 Posted May 13, 2019 Author Posted May 13, 2019 Ok, we have added today Reddis, but now it is set as data storage method, not cache method in ACP. Is that correct config?
bfarber Posted May 13, 2019 Posted May 13, 2019 When you set it as data store method, there's no need to use it as cache method as well (you'd just be copying the stuff from the data store, within Redis, to the cache store, also within Redis, effectively storing this data twice in the same engine).
wegorz23 Posted May 14, 2019 Author Posted May 14, 2019 Ok, we have now redis, cloudflare, next step will be converting all tables to innodb. Im also wondering about elasticsearch. We have default mysql search now. Could it help to switch for elasticsearch?
bfarber Posted May 15, 2019 Posted May 15, 2019 Well, there's a balance. With ElasticSearch, the software has to submit an external request to push content into the search index, so simply using ElasticSearch may not necessarily "speed up posting". If the issue you're facing, however, is that there is a lot of database contention (particularly with the search index table), then it may help. My advice would be to get all tables to InnoDB first and then re-assess. This may solve your concerns all on its own.
ASTRAPI Posted May 15, 2019 Posted May 15, 2019 My comment above is answering the question if Elastic search will help in general and not for the specific issue..... Just to avoid any misunderstanding ...
wegorz23 Posted May 16, 2019 Author Posted May 16, 2019 20 hours ago, bfarber said: My advice would be to get all tables to InnoDB first and then re-assess. This may solve your concerns all on its own. Ok, we will make it first, and then back here :) Thank you. 11 hours ago, ASTRAPI said: My comment above is answering the question if Elastic search will help in general and not for the specific issue..... Just to avoid any misunderstanding ... General speed is also very important. We have 8-10 seconds on google analytics now, so its to much for users I think.
Day_ Posted May 18, 2019 Posted May 18, 2019 On 5/13/2019 at 2:16 PM, bfarber said: When you set it as data store method, there's no need to use it as cache method as well (you'd just be copying the stuff from the data store, within Redis, to the cache store, also within Redis, effectively storing this data twice in the same engine). Would it be better to use... MySQL as Datastore and Redis as Caching Method Or Redis as Datastore and no caching
bfarber Posted May 19, 2019 Posted May 19, 2019 Honestly, for an average sized site you would probably be best off not even bothering adding Redis. There's a balance at the end of the day for everything - Redis can be amazing as your site grows and you have a lot of traffic, but you can be better off with less traffic not even using Redis (it's one less socket connection on the backend, not having to duplicate data into memory, etc.). Basically what I'm getting at is there isn't a one size fits all answer to the question. But having said that, if you have a large site and busy site, I would probably just use Redis (while you are thinking in terms of data store and no caching, that's misleading - most of the caching within the software is effectively duplicating the data store contents, so it simply wouldn't make sense to do that when the data is already being stored in Redis to begin with). Your next best option in most cases is probably to use MySQL as a data store, and not bother with caching.
wegorz23 Posted October 25, 2019 Author Posted October 25, 2019 On 5/19/2019 at 10:49 PM, bfarber said: Honestly, for an average sized site you would probably be best off not even bothering adding Redis. There's a balance at the end of the day for everything - Redis can be amazing as your site grows and you have a lot of traffic, but you can be better off with less traffic not even using Redis (it's one less socket connection on the backend, not having to duplicate data into memory, etc.). We have ~15 000 users per day (from Google Analytics) is it more or less? 🙂  Ok, anyway problem with long post adding still exists. We have tried elastic search, but it crushes every few hours. My admin made some script to make it up, but it also crushes Activity tab, and makes some few other problems, so now we have switched it off. We have older version of Ubuntu on our container on server, maybe that's why?
ASTRAPI Posted October 25, 2019 Posted October 25, 2019 We just guessing .... Hard to say without checking. It can be a simple mysql query, a bad server configuration or something else that use all the resources on the server e.t.c
Lucas James Posted October 25, 2019 Posted October 25, 2019 @wegorz23 A quick tip, recheck your firewall configuration and its rules; as one of my client had similar kind of situation on their dedicated server and the actual cause was the misconfigured CSF firewall, the ModSecurity ruleset and mod_evasive Apache module which I fixed. Anyways, I agree with @ASTRAPI – there could be many possibilities unless analyzing the backend.
wegorz23 Posted October 28, 2019 Author Posted October 28, 2019 Yeah, I know that it could be many things, but there are always some general good practices 🙂
wegorz23 Posted November 6, 2019 Author Posted November 6, 2019 Ok, we have InnoDB now, but adding posts is still broken - it takes ~10-30 seconds 😞 But it depends in which topic. I have also one more thing from my server admin: we have only 4gb swap on this machine, and 32gb of RAM. In years before there was the rule that swap should be times two of RAM. It can be related? 4gb of swap is not to small?
Recommended Posts
Archived
This topic is now archived and is closed to further replies.