Randy Calvert Posted March 31 Posted March 31 (edited) 1 hour ago, Adlago said: @Randy Calvert Is there a configuration that creates an issue described by the author of the topic? In the errors described by the OP where he's copy/pasted an IPB trace, each one has had 3rd party code or resources. He now says each 3rd party resource is disabled and now he's running out of system resources. Quote After many investigations, my server resources (CPU and RAM) suddenly skyrocketed and crashed the server. I want to record logs to a text file for investigation purposes, Please help me better understand which log I should record for the root cause undersatnding. In addition, he's posted the following as multiple entries from the PHP log: Quote 29-Mar-2024 22:31:09] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 6 total children Having a poorly configured php-fpm configuration CAN lead to system stability issues when the max number of instances that are able to be handled by PHP is reached and it can lead to a dramatic increase in the system load average. Thus the suggestion to optimize the config and/or increase system resources. I'm not sure how checking an end user's security headers presented will change system utilization or diagnose code level issues. So I've intentionally not commented on the suggestion you've presented. I assume you have an idea about something outside of it being a configuration issue or IPB code or 3rd party resource issue that I've not considered. (Which is great... the more ideas to try the better!) Edited March 31 by Randy Calvert LiveG 1
LiveG Posted March 31 Author Posted March 31 (edited) @Randy Calvert Thank you. This is are the parameter: pm.max_children = 50000 pm.start_servers = 2 pm.min_spare_servers = 3 pm.max_spare_servers = 3 pm.max_requests = 500 How should i know the recommended value? Edited March 31 by LiveG
Randy Calvert Posted March 31 Posted March 31 7 hours ago, LiveG said: @Randy Calvert Thank you. This is are the parameter: pm.max_children = 50000 pm.start_servers = 2 pm.min_spare_servers = 3 pm.max_spare_servers = 3 pm.max_requests = 500 How should i know the recommended value? How much RAM do you have available for your server? You would want to look at how much memory an FPM process uses and divide that by about 75-80 pct of your total. (Make sure to standardize both numbers so they’re same type such as both in KB or bytes.) That would reflect how many servers you can have running in total without going into a OOM (Out of Memory) situation. Once you share the total RAM, also share an FPM line from top report.
LiveG Posted March 31 Author Posted March 31 (edited) I have total 10240 MB of RAM. [31-Mar-2024 15:00:33] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 32 children, there are 0 idle, and 57 total children [31-Mar-2024 15:28:46] NOTICE: exiting, bye-bye! Something i don't understand, the command you gave me show this path: /etc/php/8.0/fpm/php-fpm.conf But when i want to edit this file - i don't see all the parameter option to edit https://ibb.co/THYjxVV i can see only process.max ( i have enlarge it from 128 to 640) On the other and, when i enter /etc/php/8.0/fpm/php-fpm.conf /pool.d/wwwconf i Can see all the parameter to edit https://ibb.co/tc7vmpJ Which file should i edit? Edited March 31 by LiveG
LiveG Posted March 31 Author Posted March 31 From the last crash: [31-Mar-2024 18:13:54] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 999 idle, and 1010 total children I increase it all values dramatically, and the site still crashing..i dont understand what is going on
Jim M Posted March 31 Posted March 31 1 minute ago, LiveG said: I increase it all values dramatically, and the site still crashing..i dont understand what is going on If you are unsure what you are doing, I would advise contracting a server administrator to assist you. Having that many items idle and children are likely creating problems in of itself.
Randy Calvert Posted March 31 Posted March 31 (edited) You don't want to increase ALL parameters drastically. You want to set the values to be in line with what your server can handle. I would recommend trying to understand what each of these parameters actually are doing here: pm.max_children = 50000 This should be your total RAM dedicated to the web server / Max child process size (look at something like "top" or "ps aux | grep fpm" and look at the size of the memory being used) pm.start_servers = 2 This is how many instances are initially spawned. So at the time Apache and FPM are started, this is how many get kicked off without anything happening. pm.min_spare_servers = 3 This is the minimum number of children in 'idle' state (waiting to process). If the number of 'idle' processes is less than this number then some children will be created. pm.max_spare_servers = 3 This is the minimum number of children in 'idle' state (waiting to process). If the number of 'idle' processes is less than this number then some children will be created. If you simply just raise all of the values, you're going to have a ton of extra processes running. Having the min and max being the same exact value also is problematic as well as it does not give you an easy way to scale processes. I would take a look at the following to understand a bit more about FPM tuning: https://tideways.com/profiler/blog/an-introduction-to-php-fpm-tuning I would also highly recommend adding a few parameters such as pm.process_idle_timeout which will end extra spawned processes that are not being used. There are a bunch more parameters available as well that can potentially optimize your experience. Checking the docs should give you a better idea of what is possible. Edited March 31 by Randy Calvert
Adlago Posted March 31 Posted March 31 This story looks more and more to me like a missing closing curly brackets ace in some code in a template causing an infinite loop and crashing the forum. @LiveG When your forum is up and running, open your forum with dev tool your browser and check the console for logged errors.
LiveG Posted April 2 Author Posted April 2 Thank you everyone. At this moment it looks like DDOS attack. i will update.
LiveG Posted April 5 Author Posted April 5 (edited) Update: it 100% DDOS attack. we will resolve it in our security infrastructure. Edited April 5 by LiveG
Adlago Posted April 5 Posted April 5 10 hours ago, LiveG said: Update: it 100% DDOS attack. we will resolve it in our security infrastructure. On 3/30/2024 at 10:56 PM, Adlago said: Test your security headers https://securityheaders.com/ and show your results You never showed a result of this test, you would have seen your vulnerability site.
Recommended Posts