Reducing the server load

[Fmoraes]

After a few days with the IPB3 noticed a considerable increase in the processing server, I got to do some testing until I found the solution.

Using mod_rewrite in ipb3, and putting. Htaccess indicated by invision:

<IfModule Mod_rewrite.c>

Options-MultiViews

RewriteEngine On

RewriteBase /

RewriteCond% (REQUEST_FILENAME)!-F

RewriteCond% (REQUEST_FILENAME)!-D

RewriteRule. / index.php [L]

</IfModule>

I had a completely absurd overload on the server load was around 0.80 0.90 jumped to 4.30 4.50 and after breaking his head a lot, I realized that the problem was that education mod_rewrite

Simply replace by:

RewriteEngine On

RewriteBase /

RewriteRule ^forum/(.*) /index.php?/forum/$1 [L]

RewriteRule ^topic/(.*) /index.php?/topic/$1 [L]

RewriteRule ^user/(.*) /index.php?/user/$1 [L]

RewriteRule ^forum-(.*)/(.*) /index.php?/forum-$1/$2 [L]

With this managed to keep the functionality of mod_rewrite while having a lower load completely

[SecondSight]

Hello ! :)

Sorry, I don't understand : where do you find the lines you suggested to modify and the .htaccess ?


I'm also having sudden high server loads since I'm using IPB3. It appears that this is a problem with httpd...

I installed PRM and it restarts httpd when there is a high server load. But this is not a solution...

I found this in the logs :

Example in PRM logs (they all look the same with a high number of processes) :

USER: nobody

PID : 9330

CMD : /usr/local/apache/bin/httpd

CPU%: 0 (limit: 65)

MEM%: 0 (limit: 25)

PROCS: 181 (limit: 150)

Here is a log example given by dcpumon :

76492

1268175301

500=0=0=0=0.002==0.001==0=

68=0=0=0=0.002==0.001==0=

dbus=0=0=0=0.002==0.001==0=

eximstats=0=0=0.443209747424567=0.002==0.001==0=

myforums=0.0549077027663019=0=1.02368875176487=14.0=/bin/gtar -c -f - -X /home/f$

mailman=0=0=0=0.002==0.001==0=

mailnull=0=0=0=0.002==0.001==0=

mysql=3.7=2.99408565601631=0=3.7=/usr/sbin/mysqld --basedir/ --datadir/var/lib/$

named=0.1=0=0=0.1=/usr/sbin/named -u named -t /var/named/chroot

nobody=21.8681796789204=3.81444987711129=0=24.0=/usr/local/apache/bin/httpd -k $

root=4.28408461015528=3.71176593630701=1.99610416775609=96.5=gzip=95.5=gzip=94.$

sshd=0=0=0=0.002==0.001==0=

unauthenticated=0=0=0.0078439575380432=0.002==0.001==0=

With the information I give here, do you think I'm having the same problem you had ?

Thank you ! :)

[SecondSight]

With this managed to keep the functionality of mod_rewrite while having a lower load completely

Hello ! :)

I'm trying it now. Up to now it's working fine, but I'll have to wait until tomorrow, when I have many members online, to know if it's really ok. :)

This is what I wrote :

<IfModule mod_rewrite.c>

Options -MultiViews

RewriteEngine On

RewriteBase /

RewriteRule ^forum/(.*) /index.php?/forum/$1 [L]

RewriteRule ^topic/(.*) /index.php?/topic/$1 [L]

RewriteRule ^user/(.*) /index.php?/user/$1 [L]

RewriteRule ^forum-(.*)/(.*) /index.php?/forum-$1/$2 [L]

</IfModule>

I don't understand what it is doing exactly. Can you explain to me ?
Also, must I remove the "Options -MultiViews" ?

Now, there is a problem with the link which is just under the green "Forums" tab. It now leads to a blank page...


Do you know how I can fix this ?

Thank you ! :)

[Fmoraes]

Using this system rewrite, you use the redirect only requests for web pages, using the method indicated by invision, all requests for images and css are redirected to index.php .. This creates an unnecessary overload

to correct the index (home page) is simple, add the line:

RewriteRule ^index /index.php?/index [L]

sorry add Options -MultiViews ... :whistle:

[SecondSight]

RewriteRule ^index /index.php?/index [L]

Sorry, this line didn't fix the problem.

It seems like the server load is divided by 3 on my server. I will check this on a longer period, tonight when I have many members.

[bfarber]

Using this system rewrite, you use the redirect only requests for web pages, using the method indicated by invision, all requests for images and css are redirected to index.php .. This creates an unnecessary overload

Actually, that's not entirely true.

RewriteCond% (REQUEST_FILENAME)!-D

RewriteCond% (REQUEST_FILENAME)!-F

These two lines in the default .htaccess tell Apache to ONLY run the mod_rewrite if the file does not physically exist on disk, and/or if the directory does not physically exist on disk (in the case of css or images, they do physically exist, so they are not routed through index.php).

I'm not entirely sure why you'd see such dramatic improvements in resource usage with the changes you've made. It's a bit interesting, and something I will push to the other devs to review as well. The main problem with the method proposed is that it doesn't take into account other applications. i.e. IP.Content, blog, gallery, and so on, and then 3rd party applications can use our furl system, so they'd need a way to support this as well. Little tricky but we'll take a look.

By the way, I'd recommend posting a resource article as well on your findings. :)

[SecondSight]

Do you know a way of fixing the broken links ?

Thank you ! :)

[Fmoraes]

On sites with large amount of access that would avoid checking the file on disk, allowing other applications such as database work more quickly.

The system of Blogs, ipContent and etc, would be simple to generate these rewrites and adapts them to third-party systems ..

RewriteRule ^index /index.php?/index [L]

Running fine on IIS 7.5

[SecondSight]

I'm using Centos 5...

[SecondSight]

I've tested it tonight for 4 hours, and the server load has significantly been reduced. You don't see the difference when you have only 100 users online (this morning) but it's obvious when you have about 600 users online.

[SecondSight]

I tried again RewriteRule ^index /index.php?/index [L] and I've had a look in error_log and here is what I found there :

Request exceeded the limit of 10 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace., referer: [url="http://myforums.com/forum/125-talks/"]http://myforums.com/forum/125-talks/[/url]

What should I do according to you ? :)

Thank you, :)

[SecondSight]

Someone told me this :

That error is because there is a loop being created. There is no problem with the amount of recursions, because if it is correct it should only redirect 1 or 2 times. There is something wrong with the code, it should not be doing over 10 redirects.

Anyone having a solution ?

Thank you ! :)

[Fmoraes]

Sorry for the delay in replying, but test the following rule:
RewriteRule ^index$ /index.php?/index [L]

Tested on Apache 2.0.63 with WampServer

.htaccess

Options -Multiviews

RewriteEngine On

RewriteBase /

RewriteRule ^index$ /index.php?/index [L]

RewriteRule ^forum/(.*) /index.php?/forum/$1 [L]

RewriteRule ^topic/(.*) /index.php?/topic/$1 [L]

RewriteRule ^user/(.*) /index.php?/user/$1 [L]

[marklcfc]

Our server load used to be between 0-1 unless it got busy, even then it would only go upto 5 on most occasions.

Now since upgrading to IPB 3, even during quiet periods the server load is never below 3. Is there something I need to do to correct this? Is there anything I need to put in .htaccess as I don't have anything other than a redirect in there...

RewriteEngine on

RewriteCond %{HTTP_HOST} ^mysite.co.uk$ [OR]

RewriteCond %{HTTP_HOST} ^www.mysite.co.uk$

RewriteRule ^/?$ "http://www.mysite.co.uk/forums" [R=301,L]

[SecondSight]

Sorry for the delay in replying, but test the following rule:

RewriteRule ^index$ /index.php?/index [L]

It's working for me. Thank you ! :)

I will test all this tonight when I have about 600/700 users online. :)

[SecondSight]

Ok. This way, I managed to have a low server load all the time there was many users online. Thanks a lot FMoraes. :)

[SecondSight]

Now, I have noticed that :
- we can't access the blogs.
- the announcements can't be read.
Anyone knows what to write in the .htacces to have the blogs and announcements work ?

Thank you ! :)

[Fmoraes]

Sorry, but I'm not the blogging system, but if you can pass the url, I ride the rewrite. ..

for announcements is simple.
RewriteRule ^forums-(.*)/(.*) /index.php?/forums-$1/$2 [L]

[SecondSight]

Hello ! :)

Blogs are like this :

http://community.invisionpower.com/blogs/

So the solution would be ?

RewriteRule ^blogs/(.*) /index.php?/blogs/$1 [L]

A blog url is like this :

It is different, so should I write one more line ?

RewriteRule ^blog/(.*)/(.*) /index.php?/blog/$1/$2 [L]

Do you think it's ok for the blogs ? I've tried it on my board and it's working. There are no errors in Apache error_log file. As for announcements, I get a blank page with the line :

RewriteRule ^forums-(.*)/(.*) /index.php?/forums-$1/$2 [L] 

Thank you ! :)

[SecondSight]

As for announcements, I get a blank page with the line :

RewriteRule ^forums-(.*)/(.*) /index.php?/forums-$1/$2 [L] 

In fact, you must delete the s with forums and use :

RewriteRule ^forum-(.*)/(.*) /index.php?/forum-$1/$2 [L] 

Now it seems like it's working.

Thanks a lot ! :)

[Fmoraes]

Do you understand how it works rsrsrsrs

Basically they are only redirected the links and other types of files do not go through index.php .. thus creating less load on the server.

Exactly, his rule for blogs is perfect, .. :)

RewriteRule ^blogs/(.*) /index.php?/blogs/$1 [L]

[SecondSight]

Do you understand how it works rsrsrsrs

I've read a little about mod_rewrite and I think I understand more or less how to do this, but at a simple level... :)

Thank you for your help ! :)