Robert Angle Posted October 5, 2020 Posted October 5, 2020 We crashed the other day. There was a burst of traffic that caused out of memory errors which forced the server into 'read only' mode and then nothing worked, and when I tried rebooting the server it simply died. I'm back up and running now, but the site seems very slooooooooow much of the time. I involved my web host, and he tweaked a bunch of things, and said under normal circumstances I should be fine with our current VPS plan, but that he recommends I add CloudFlare to my site. I've done some reading about CloudFlare, and I am trying to wrap my head around what it actually does. It sounds like it serves cached static content straight from it's servers instead of my server. I am guessing this means images, pages, etc, that don't generally get updated. Certainly this wouldn't mean dynamic content that gets updated with reactions, replies, new posts, etc, right? Obviously I am concerned that visitors are getting up-to-date content when perusing the community. Another concern is that I heard it will end up messing up IP addresses. In other words, the IP address that the system logs will be that of CloudFlare's and not the ISP of the user connecting to the forum. If this is true, I would think that this would break all kinds of functionality of the software. In addition to possibly considering CloudFlare, I have been looking into 2 other options to try. 1) Archive posts over XX years old. We're currently sitting on 37,000 topics and 750,000 replies. If I chose to archive topics over 5 years old, about half of these would get moved to another table. These would be no longer editable and won't show up in searches, which would be a bummer. But, would it save load times? 2) Install and use Redis. Currently I use the file system as a datastore and no caching method at all. I haven't upgraded to 4.5 yet but it seems that Redis is the way to go moving forward. I don't know if I would benefit from using it though. Would I? Any feedback would be greatly appreciated. As soon as I address this speed issue and check that my 3rd part plugins are compatible with 4.5 then I will be upgrading. - Bob
Lucas James Posted October 5, 2020 Posted October 5, 2020 10 hours ago, Robert Angle said: Any feedback would be greatly appreciated. VPS are low-powered with limited/less resources. Instead, get a dedicated server with good specs at reasonable price Optimize the server security/speed for PHP, Nginx, PHP-FPM, MariaDB, etc Cloudflare Pro with optimized settings and WAF enabled Redis Web-API for outgoing Emails 10 hours ago, Robert Angle said: I am concerned that visitors are getting up-to-date content when perusing the community. It isn't the issue with Cloudflare, so you're users will see the live updates without cache problem. 10 hours ago, Robert Angle said: I heard it will end up messing up IP addresses. In other words, the IP address that the system logs will be that of CloudFlare's and not the ISP of the user connecting to the forum. If this is true, I would think that this would break all kinds of functionality of the software. There's already a setting in IPS software to compensate that. SammyS 1
bfarber Posted October 5, 2020 Posted October 5, 2020 Does your host have any more information on what caused the server crash/out of memory? Those suggestions can certainly help with performance (and pretty significantly), but if there's any reason to believe something happened at the software level we'd also be interested in getting some more information.
Robert Angle Posted October 6, 2020 Author Posted October 6, 2020 11 hours ago, bfarber said: Does your host have any more information on what caused the server crash/out of memory? Those suggestions can certainly help with performance (and pretty significantly), but if there's any reason to believe something happened at the software level we'd also be interested in getting some more information. On September 30th all I was told was that there was a spike in traffic that caused the server to go into read-only mode. This crash my site. When I tried to reboot the server it died and I had to contact them to get it up and running again. Today I had another problem. Memory usage spiked, causing connection time-outs. Turns out there was a bruteforce attack on one of my other domains (a wordpress site). He disabled the login for wordpress to stop it and then everything went back to normal. So it seems that it is outside forces causing the problems, and not the server itself or the software. Even still, I need some education on Redis and Cloudflare. Are these options I should be employing?
bfarber Posted October 6, 2020 Posted October 6, 2020 We recommend Cloudfront from Amazon rather than Cloudflare. It behaves similarly and can improve performance (it is leveraged automatically for all of our Community In The Cloud accounts). Redis can improve performance for many users, but it's not a guarantee. It depends on various factors, however generally speaking if you are looking to improve performance I would definitely recommend trying it out if it's an option in your hosting environment.
Runar Posted October 22, 2020 Posted October 22, 2020 On 10/6/2020 at 3:40 PM, bfarber said: We recommend Cloudfront from Amazon rather than Cloudflare. It behaves similarly and can improve performance (it is leveraged automatically for all of our Community In The Cloud accounts). Any specific reason why you recommend Cloudfront over Cloudflare?
CoffeeCake Posted October 22, 2020 Posted October 22, 2020 3 hours ago, Runar said: Any specific reason why you recommend Cloudfront over Cloudflare? One flares at the end, the other is mostly in the front.
Runar Posted October 22, 2020 Posted October 22, 2020 (edited) 3 minutes ago, Paul E. said: One flares at the end, the other is mostly in the front. Edited October 22, 2020 by Runar
ptprog Posted October 30, 2020 Posted October 30, 2020 On 10/6/2020 at 5:50 AM, Robert Angle said: So it seems that it is outside forces causing the problems, and not the server itself or the software. Even still, I need some education on Redis and Cloudflare. Are these options I should be employing? I don't know IPS implementation in detail, but if you are having out of memory problems, I suspect that adding Redis (in the same server) will only make things worse, as it will be yet another service contending for memory. You probably need to review the software limits (memory, number of connections, etc.) to make some requests fail when there are spikes, instead making the entire server go down. Cloudflare should help reducing the load on the server, and maybe it will also help in case of attacks. (I terms of performance I also found Cloudfront to be better, and it would be my choice for a pure CDN, but I'm not sure if Cloudfront is comparable to Cloudflare in terms of security features.) Archiving posts will likely reduce memory pressure on DB side. I assume it will at least reduce the size of the search index, so less data that needs to be cached in memory.
CheersnGears Posted October 30, 2020 Posted October 30, 2020 On 10/4/2020 at 9:11 PM, Robert Angle said: We crashed the other day. There was a burst of traffic that caused out of memory errors which forced the server into 'read only' mode and then nothing worked, and when I tried rebooting the server it simply died. I'm back up and running now, but the site seems very slooooooooow much of the time. I involved my web host, and he tweaked a bunch of things, and said under normal circumstances I should be fine with our current VPS plan, but that he recommends I add CloudFlare to my site. I've done some reading about CloudFlare, and I am trying to wrap my head around what it actually does. It sounds like it serves cached static content straight from it's servers instead of my server. I am guessing this means images, pages, etc, that don't generally get updated. Certainly this wouldn't mean dynamic content that gets updated with reactions, replies, new posts, etc, right? Obviously I am concerned that visitors are getting up-to-date content when perusing the community. Another concern is that I heard it will end up messing up IP addresses. In other words, the IP address that the system logs will be that of CloudFlare's and not the ISP of the user connecting to the forum. If this is true, I would think that this would break all kinds of functionality of the software. In addition to possibly considering CloudFlare, I have been looking into 2 other options to try. 1) Archive posts over XX years old. We're currently sitting on 37,000 topics and 750,000 replies. If I chose to archive topics over 5 years old, about half of these would get moved to another table. These would be no longer editable and won't show up in searches, which would be a bummer. But, would it save load times? 2) Install and use Redis. Currently I use the file system as a datastore and no caching method at all. I haven't upgraded to 4.5 yet but it seems that Redis is the way to go moving forward. I don't know if I would benefit from using it though. Would I? Any feedback would be greatly appreciated. As soon as I address this speed issue and check that my 3rd part plugins are compatible with 4.5 then I will be upgrading. - Bob You really need to know what ran out of memory... was it Apache? PHP? SQL? It's possible your SQL resources are too low and causing a drag on performance. If your host is managing this, more than likely they have your resources set pretty low so they can oversell your server. It's mostly a thing of the past, but it is still possible for Apache to go and eat up all of your memory if you have high traffic.... if the maximum amount of memory Apache plus the maximum amount of memory SQL are configured to use is greater than the amount of memory available, then you're setting yourself up for a crash. My first instinct would be to look at SQL... but you may not have that level of access depending on who you host through.
Recommended Posts