Great Results with Nginx Cache

[.Nuno.]

Hi,

Following this topic about varnish I decided to try an alternative.
Since i was running nginx this was easy step.

A quick note ... this is a learning by progress idea and you must evaluate all the benefits of using it.

I have this working in my board with a page load time decrease of + 60% tested with Gomez Web Site Performance Test for guests, since this only cache content for unregistered members.

You only need nginx + fastcgi / proxy* + ssi (* I didn't test with proxy_cache but it may work)

Somewhere in you http block configuration add:

### Map Mobile

map $http_user_agent $is_mobile {

  default 0;

  ~*android|ip(hone|od)|windowss+(?:ce|phone) 1;

  ~*symbian|sonyericsson|samsung|lg|blackberry 1;

}

You may add more devices if you want. At least match the ones in your agent mapping mobile theme. Next, add your cache path to the http block:

### FastCGI Cache

fastcgi_cache_path /var/cache/nginx/board levels=1:2 keys_zone=ONE:500m inactive=60m max_size=500m;

Next, add to your fastcgi location:

location ~ .php$ {

  .......................................

  your fastcgi configuration

  .......................................

  ssi on;

  ssi_silent_errors on;

  fastcgi_cache		 ONE;

  fastcgi_cache_valid 200 302 15m;

  fastcgi_cache_valid 301	 15m;

  fastcgi_cache_valid any	 1m;

  fastcgi_cache_min_uses 1;

  fastcgi_cache_use_stale error timeout invalid_header http_500;

  fastcgi_cache_key "$scheme$request_method$host$request_uri";

  fastcgi_no_cache $cookie_member_id $cookie_mobilebrowser $cookie_guestskinchoice $cookie_language $is_args $is_mobile;

  fastcgi_cache_bypass $cookie_member_id $cookie_mobilebrowser $cookie_guestskinchoice $cookie_language $is_args $is_mobile;

  fastcgi_ignore_headers Cache-Control Expires Set-Cookie;

  add_header X-Cache $upstream_cache_status;

}

Now, for your topics views counter to work, we will use SSI. First create a php file in your root with this content:

<?

header("Expires: Mon, 26 Jul 1997 05:00:00 GMT");

header("Cache-Control: no-cache");

header("Pragma: no-cache");

header("X-Accel-Expires: 0");

if (is_numeric($_GET["tid"])) {

require_once( './conf_global.php' );

$link = mysql_connect($INFO['sql_host'], $INFO['sql_user'], $INFO['sql_pass']);

@mysql_select_db($INFO['sql_database'], $link);

@mysql_query("INSERT INTO topic_views (views_tid) VALUES ('" . $_GET['tid'] . "')");

mysql_close($link);

}

?>

You can named what you want, the inclusion bellow needs to match the path & file name. Next, in you Topic View -> topicViewTemplate template add this to the top:

<if test="!$this->memberData['member_id']">

<!--# include virtual="/path/to/your/file.php?tid={$topic['tid']}" -->

</if>

Now test and show the results. What needs to be done:

• Purging content when we have new posts

I have done this with varnish and I'm testing a few ideas, but for now we must rely in cache_valid. I'm thinking about this two modules: https://github.com/p...inx-cache-purge https://github.com/F...ngx_cache_purge The second one will only purge one page per request. The first is a bash script that will allow us to purge more than one page per action and that's what I need, but I haven't tested yet; ex:

nginx-cache-purge "/topic/1-welcome/*" /var/cache/nginx/board

note the *

• A way to add/remove forum sessions with SSI so we can get the correct users online.
  

We already know that we can add SSI to call dynamic content inside cache files. Now I only need someone to discover a way we can add/remove session for forum users.
We can add a ssi file to globalTemplate like I did for views counter. I only need someone with more IPB knowledge than me to find a way we can get sessions to work.

Final notes:

For the files you don't want to cache you must add a new header: X-Accel-Expires:

"The X-Accel-Expires header field sets caching time of a response in seconds. The value 0 disables to cache a response."
http://nginx.org/en/...tml#proxy_cache


This is working for me and should work for you. You should test this before applying to a live board.
I'm not responsible for bad behavior this may cause to you board.

[Enkidu]

great post Nuno

I still struggle with nginx-admin for cpanel. I can't see any positve results with it. will follow your post and hopefully it will work for me :)

thanks

[.Nuno.]

Enkidu ... test and show your results.
I'm not using cpanel, sorry.
I have very good results with minimal effort ... just need to solve the sessions problem.

[Luis Manson]

.nuno. how do you manage IPB error pages? like server overload

[Grumpy]

great post Nuno

I still struggle with nginx-admin for cpanel. I can't see any positve results with it. will follow your post and hopefully it will work for me :smile:

thanks

Unless you nest the scope properly, I'm sure Nuno's setting will wreak havoc in cpanel.

[.Nuno.]

.nuno. how do you manage IPB error pages? like server overload

Hi Luis,

Errors are handled according to these settings:

fastcgi_cache_valid 200 302 15m;

fastcgi_cache_valid 301	    15m;

fastcgi_cache_valid any	    1m;

fastcgi_cache_use_stale        error timeout invalid_header http_500;

Header with 200, 301 and 302 are cache during 15m. Other are cached for 1m
fastcgi_cache_use_stale tell us that in case of an error serves a old version of the cache if it exists.

So, if we get a 4xx/5xx error, nginx try to serve a valid cached version (200). If doesn't exists, you get a 4xx/5xx according to the header sent from IPB.

http://nginx.org/en/docs/http/ngx_http_fastcgi_module.html

Regarding server overload, like I said before, you get the results according the header sent by IPB. If a server overload sen't a 503 error, nginx serves the cached version until it expires.

[Luis Manson]

thanks nunno, the overload error is sending an http 401 error :( maybe it could be added to this config?


also, for a site on production, would you use this or varnish? (im on nginx alread)

[Luis Manson]

i can get it to work..maybe its related to:

Yes. But be aware of Cache-Control header sent by your backend. (

http://wiki.nginx.org/HttpProxyModule#proxy_cache : The cache honors

backend's "Expires", "Cache-Control: no-cache", and "Cache-Control:

max-age=XXX" headers since version 0.7.48. Since version 7.66, "private"

and "no-store" are also honored.)

also i have a session_id cookie, not sure where it comes from...

[maddog107_merged]

How did your performance compare to the varnish setup you tried?

[.Nuno.]

thanks nunno, the overload error is sending an http 401 error :sad: maybe it could be added to this config?

also, for a site on production, would you use this or varnish? (im on nginx alread)

Header other than 200, 301 and 302 are only cached for 1m. If you prefer to avoid this, I would remove the following line:

fastcgi_cache_valid any	    1m;

Without this, only headers with code 200, 301 and 302 are changes ... all other are bypassed. I'm using this in production and I prefer it over varnish.

i can get it to work..maybe its related to:

Yes. But be aware of Cache-Control header sent by your backend. (

http://wiki.nginx.org/HttpProxyModule#proxy_cache : The cache honors

backend's "Expires", "Cache-Control: no-cache", and "Cache-Control:

max-age=XXX" headers since version 0.7.48. Since version 7.66, "private"

and "no-store" are also honored.)

All these headers aren't passed to the backend:

fastcgi_ignore_headers Cache-Control Expires Set-Cookie;

also i have a session_id cookie, not sure where it comes from...

It looks like you're using proxy. This can be a different setup. If you are using nginx to proxy to apache, you should try the following: Add theses header:

proxy_set_header   Accept-Encoding  "";

proxy_hide_header  Vary;

Disable Expire and Deflate modules from apache and ensure IPB gzip is disabled too.
Session Cookie aren't passed to the backend.

How did your performance compare to the varnish setup you tried?

I prefer this setup and in my environment it's faster than varnish.
The configuration it's simple and if you have more sites in the same IP, with varnish all need to pass all the requests, even if you don't cache it.
For the static content, it doesn't make sense to use varnish since nginx is faster too.

[Dmacleo]

you must be using nginx in place of apache?
you probably are not on a cpanel server then correct?

[.Nuno.]

you must be using nginx in place of apache?

you probably are not on a cpanel server then correct?

Correct ... I use nginx + fpm only.

[Dmacleo]

I'm using it on cpanel so it kind of has to act like a proxy but it seems to help.
debated breaking cpanel/easyapache and using only nginx but its kind of pita to do that.

[Luis Manson]

this is weird, not sure how is your setup, but mine is a bit messy and kinda simple too

www.MainSite.com - statics1.MainSite.com - statics2.MainSite.com
statics domains are used for CSS/JS/Images

The headers i have now are:

curl -I http://www.MainSite.com/forum/69-motorola/

HTTP/1.1 200 OK

Server: nginx

Date: Tue, 12 Jun 2012 12:16:30 GMT

Content-Type: text/html;charset=UTF-8

Connection: keep-alive

Vary: Accept-Encoding

Set-Cookie: /session_id=978b9385d1112258f59c31b470b2d644; path=/; domain=www.MainSite.com; httponly

Set-Cookie: /modtids=%2C; path=/; domain=www.MainSite.com

X-Cache: BYPASS

varnish kinda worked, but i did not want another layer of complexity so i opted for nginx...
also varnish beresp.http.X-Purge 'Barneys Girlfriend'ed me about sintaxys

[.Nuno.]

Can you paste you configuration?

[Luis Manson]

sure, sent via PM

[.Nuno.]

Here is a configuration that SHOULD work:

FILE NAME: /etc/nginx/php5_params

---------------------------------------------------------------------

access_log off;

log_not_found off;

try_files $uri = 404;

fastcgi_split_path_info ^(.+.php)(/.+)$;

include fastcgi_params;

fastcgi_param  PATH_INFO	    $fastcgi_path_info;

fastcgi_index index.php;

fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;

fastcgi_pass php5-fpm;

---------------------------------------------------------------------


## Upstream to FPM

upstream php5-fpm  {

  server unix:/var/run/php5-fpm.sock;

}

## FastCGI Cache

fastcgi_cache_path   /var/cache/nginx/one  levels=1:2

					 keys_zone=one:500m

					 inactive=60m

					 max_size=500m;


### Map Mobile

map $http_user_agent $is_mobile {

  default 0;

  ~*android|ip(hone|od)|windowss+(?:ce|phone) 1;

  ~*symbian|sonyericsson|samsung|lg|blackberry 1;

}


## Server Block

server {

	    listen   80;

	    server_name www.example.com;

	    root /srv/www/example.com/public;

	    index index.php index.html;

	    access_log  /var/log/nginx/example-access.log  main;

	    error_log  /var/log/nginx/example-error.log;

	    ## Deny .htaccess

	    location ~ /. {

		    access_log off;

		    log_not_found off;

		    deny all;

	    }

	    ## Disable php file execution

	    location ~* ^/(?:cache|hooks|public/style_css|public/style_images|uploads)/.*.(?:pl|php[345]*)$ {

		    access_log off;

		    log_not_found off;

		    return 404; # or 403

	    }


	    ## Main location

	    location / {

		    try_files $uri $uri/ /index.php;

	    }

	    ## Admin

	    location /admin/ {

		    access_log off;

		    log_not_found off;

		    try_files $uri $uri/ /admin/index.php;

		    location ~ .php$ {

			    include php5_params;

		    }

	    }


	    ## PHP

	    location ~ .php$ {

		    include php5_params;

		    ssi on;

		    ssi_silent_errors on;

		    fastcgi_cache		 one;

		    fastcgi_cache_valid   200 301 302  15m;

		    fastcgi_cache_use_stale error  timeout invalid_header http_500;

		    fastcgi_cache_key "$scheme$request_method$host$request_uri";

		    fastcgi_no_cache $cookie_member_id $cookie_mobilebrowser $cookie_guestskinchoice $cookie_language $is_args $is_mobile;

		    fastcgi_cache_bypass $cookie_member_id $cookie_mobilebrowser $cookie_guestskinchoice $cookie_language $is_args $is_mobile;

		    fastcgi_ignore_headers Cache-Control Expires Set-Cookie;

		    add_header X-Cache $upstream_cache_status;

	    }

}

Note: you must disable gzip in IPB and make sure nginx do it for you.

[Dmacleo]

Luis are you on cpanel?
if so would you let us know how this works?
thanks.

[Luis Manson]

no, i mess manage the server by hand

[Dmacleo]

LOL thanks :)

[Elly]

Bookmarked. We're using nginx/fpm. Gonna test this out!

[Luis Manson]

i will just report something, but dont take it as granted since nothing in this thread is working FOR ME, nuno helped me A LOT but we were unable to get a solution for my problem... i have yet to try the config he posted.

basically using the try_files posted before im unable to edit ip.content articles...im posting here in case someone el faces the same situation..


about my problem: the cache works for users too, not just guest, which causes some users being shown as others... sessions are mixing XD

[.Nuno.]

Hi Lu

[Luis Manson]

i know nunno, just wanted to note the problem with the "edit" button, but also stated clearly that i have a wery rare problem which may be related to this...
PS: the settings you sentme did not make any difference...but at least they look prettier than mine LOL

[Dmacleo]

when you say can't edit you mean editor won't load or the edit won't save?