Great Results with Nginx Cache

[maddog107_merged]

I have to do this from CMD line (since I cant modify my HOSTS file since its on a live server).
ab -n 1 -v 2 -H 'Host: www.bellazon.com' http://192.168.50.2/main/index.php



Benchmarking 192.168.50.2 (be patient)...INFO: POST header ==
---
GET /main/index.php HTTP/1.0
Host: www.bellazon.com
User-Agent: ApacheBench/2.3
Accept: */*


---
LOG: header received:
HTTP/1.1 200 OK
Server: nginx/1.2.1
Date: Wed, 04 Jul 2012 21:21:59 GMT
Content-Type: text/html;charset=UTF-8
Connection: close
X-Powered-By: PHP/5.4.4
Set-Cookie: session_id=489dbfd9cb926aa651feeed61cd62581; path=/main; domain=.bellazon.com; httponly
Cache-Control: no-cache,must-revalidate, max-age=0
Expires: Tue, 03 Jul 2012 21:21:56 GMT
Pragma: no-cache
X-Cache: HIT



If I do the full URL in Chrome/FF I get BYPASS and no cache file is created for this request

• 
  Cache-Control:
  no-cache,must-revalidate, max-age=0
• 
  Connection:
  keep-alive
• 
  Content-Encoding:
  gzip
• 
  Content-Type:
  text/html;charset=UTF-8
• 
  Date:
  Wed, 04 Jul 2012 21:23:23 GMT
• 
  Expires:
  Tue, 03 Jul 2012 21:23:23 GMT
• 
  Pragma:
  no-cache
• 
  Server:
  nginx/1.2.1
• 
  Set-Cookie:
  session_id=d56fed6371872018222ce7199eb641c0; path=/main; domain=.bellazon.com; httponly
• 
  Transfer-Encoding:
  chunked
• 
  X-Cache:
  BYPASS
• 
  X-Powered-By:
  PHP/5.4.4
  

[maddog107_merged]

hmm k, sorry I should have been using CURL.


[root@BZ-1 ~]# curl -I -H 'Host: www.bellazon.com' http://192.168.50.2/
HTTP/1.1 200 OK
Server: nginx/1.2.1
Date: Wed, 04 Jul 2012 21:27:24 GMT
Content-Type: text/html;charset=UTF-8
Connection: keep-alive
X-Powered-By: PHP/5.4.4
Set-Cookie: session_id=962ed2cd20266c9e212fb956ac16aa31; path=/main; domain=.bellazon.com; httponly
Cache-Control: no-cache,must-revalidate, max-age=0
Expires: Tue, 03 Jul 2012 21:27:24 GMT
Pragma: no-cache
X-Cache: MISS

[root@BZ-1 ~]# curl -I -H 'Host: www.bellazon.com' http://192.168.50.2/
HTTP/1.1 200 OK
Server: nginx/1.2.1
Date: Wed, 04 Jul 2012 21:27:26 GMT
Content-Type: text/html;charset=UTF-8
Connection: keep-alive
X-Powered-By: PHP/5.4.4
Set-Cookie: session_id=962ed2cd20266c9e212fb956ac16aa31; path=/main; domain=.bellazon.com; httponly
Cache-Control: no-cache,must-revalidate, max-age=0
Expires: Tue, 03 Jul 2012 21:27:24 GMT
Pragma: no-cache
X-Cache: HIT

[maddog107_merged]

Actually nm it still didnt work :(


[root@BZ2 one]# head 7/58/347ac9565f53feee2e706e5cffec2587
~

[maddog107_merged]

Sorry guys I am losing my mind today, it should be lowercase 'i' in the curl command.

CHROME

Cache-Control:no-cache,must-revalidate, max-age=0

Connection:keep-alive

Content-Encoding:gzip

Content-Type:text/html;charset=UTF-8

Date:Wed, 04 Jul 2012 21:45:41 GMT

Expires:Tue, 03 Jul 2012 21:45:41 GMT

Pragma:no-cache

Server:nginx/1.2.1

Set-Cookie:session_id=1cdbd3eb5c3e8f657d4fe02e785db9f4; path=/main; domain=.bellazon.com; httponly

Transfer-Encoding:chunked

X-Cache:MISS

X-Powered-By:PHP/5.4.4


Cache-Control:no-cache,must-revalidate, max-age=0

Connection:keep-alive

Content-Encoding:gzip

Content-Type:text/html;charset=UTF-8

Date:Wed, 04 Jul 2012 21:46:17 GMT

Expires:Tue, 03 Jul 2012 21:45:41 GMT

Pragma:no-cache

Server:nginx/1.2.1

Set-Cookie:session_id=1cdbd3eb5c3e8f657d4fe02e785db9f4; path=/main; domain=.bellazon.com; httponly

Transfer-Encoding:chunked

X-Cache:HIT

X-Powered-By:PHP/5.4.4

CURL

0	 0	0	 0	0	 0	 0	 0 --:--:-- --:--:-- --:--:--	 0HTTP/1.1 200 OK

Server: nginx/1.2.1

Date: Wed, 04 Jul 2012 21:46:42 GMT

Content-Type: text/html;charset=UTF-8

Transfer-Encoding: chunked

Connection: keep-alive

X-Powered-By: PHP/5.4.4

Set-Cookie: session_id=1cdbd3eb5c3e8f657d4fe02e785db9f4; path=/main; domain=.bellazon.com; httponly

Cache-Control: no-cache,must-revalidate, max-age=0

Expires: Tue, 03 Jul 2012 21:45:41 GMT

Pragma: no-cache

X-Cache: HIT

and only ONE cache file. :smile:


So why does /main/index.php do BYPASS?

[.Nuno.]

If I curl -I [url] initially, X-Cache header is MISS. If I do it again, it's a HIT. However on the same URL in my browser, will be MISS. Thus I think cookie is making it have separate cache for each session_id?

Hi,

Can you show us the header from the binary file in nginx cache?
Are the KEYS the same in booth requests?
Do you have GZIP disabled in IPB?

Thanks

[Luis Manson]

Hi nuno, how is the stae of this? right now im with varnish and want to give it another try... but to be honest varnish is doing a god job

[Dmacleo]

what are you using to get that graph?

[.Nuno.]

Hi Luis,

I'm using nginx cache without purge successfully. Just cache pages I need for some time.

Here are the relevant configuration:

fastcgi_cache_path /var/cache/nginx/DEMO levels=1:2 keys_zone=DEMO:500m inactive=60m max_size=500m;


# Map Mobile: Default -> no cache if mobile

map $http_user_agent $is_mobile {

  default 0;

  ~*android|ip(hone|od|ad)|windows\s+(?:ce|phone) 1;

  ~*symbian|sonyericsson|samsung|lg|blackberry 1;

}


# Map Uri: Default -> no cache

map $request_uri $is_uri {

  default	 1;

  ~^/user	 0;

  ~^/members 0;

  ~^/blogs	 0;

  ~^/membermap 0;

  ~^/links/$ 0;

  ~^/gallery/$ 0;

}

# Map Cookies: Default -> no cache if cookie

map $http_cookie $is_cookie {

  default			 0;

  ~member_id=[1-9]\d* 1;

  ~mobilebrowser	 1;

  ~guestskinchoice	 1;

  ~language			 1;

}


fastcgi_cache DEMO;

fastcgi_cache_valid 200 60m;

fastcgi_cache_use_stale error timeout invalid_header http_500;

fastcgi_cache_key "$scheme$request_method$host$request_uri";

fastcgi_no_cache $is_cookie $is_args $is_mobile $is_uri;

fastcgi_cache_bypass $is_cookie $is_args $is_mobile $is_uri;

fastcgi_ignore_headers Cache-Control Expires Set-Cookie;

add_header X-Cache $upstream_cache_status;

[Luis Manson]

what are you using to get that graph?

Its a monitoring app called Zabbix, open source :smile:
to make it work with varnish i have small script on the server to get those values and send them to the server, nothing hard

Hi Luis,

I'm using nginx cache without purge successfully. Just cache pages I need for some time.

Thanks, i will try in the weekend again and if it wants to work

i see you dont "map" topics or forums, that means they are not cached?

why did you add this URL mapping?

[Dmacleo]

will check it out, thanks

[.Nuno.]

i see you dont "map" topics or forums, that means they are not cached?

why did you add this URL mapping?

Yes, at this time I'm not caching topics and forums.
The foruns are safe to cache but if you apply this to topics you will lose topic count, unless you use ssi with a script that updates the counter.

I haven't done this because lack of time a some lazyness :smile:

I use uri mapping to have the guratee that only these urls are cached. I dont want/need to cache everything

Edit: Here is a less restrictive url mapping:

# Map Uri: Default -> no cache

map $request_uri $is_uri {

  default       1;

  ~^/links/link/ 1;

  ~^/links/goto/ 1;

  ~^/gallery/image/ 1;

  ~^/user/      0;

  ~^/members/   0;

  ~^/blogs/     0;

  ~^/membermap/ 0;

  ~^/links/     0;

  ~^/gallery/   0;

  ~^/forum/     0;

}

The only urls I'm not caching are topics/images/links which have view counters

[Luis Manson]

got it, thanks!

[xnetco]

Has anyone tried the nginx micro cache? We've been trailing this on some vBulletin sites for the past year and seen some fantastic results on large traffic spikes. We had a client linked directly from BBC News last week and the site didn't go down.

Tim

[maddog107_merged]

Microcaching is just caching everything for a few seconds, in this thread we are discussing caching forever until it needs to be purged by an update or what not. So essentially its the same thing, except our long term (macro?) caching is better as you never have to regenerate a request.

Or is there something I am missing?

[Dmacleo]

on a dynamic site thats active wondering if micro would have more benefits. sure each site has different needs though, whats good for one not for another.

[Luis Manson]

i think this could work just on some specific pages, like if your site has a VERY popular topic for a while, since visits to other areas would get purged before some one else request the same page

[yacenty]

can you confirm that such a solution is worth to implement and is working fine?

[Grumpy]

can you confirm that such a solution is worth to implement and is working fine?

Are you talking about the entire conversation about nginx? Or are you talking about microcaching mentioned right above?

[yacenty]

caching in nginx, because I'm using nginx from 3 years and just now I'm considering next step of caching especially there are about 2k users online in the peak time.

[.Nuno.]

Hi,

The easiest way is to use microcaching for guests for 5, 10 or 15 min. Just find your post rate and adjust the time for the better performance/fresh content.

You'll get good results just by fine tuning nginx.

Full cache with purge isn't easy, mainly because you have some ulr's variation.

Just go with microcaching for now and test.

[Parisian]

This is an excellent guide but it does have a few problems. Here are my modifications:

ipb_args.conf

map $http_user_agent $is_mobile {
        default 0;
        ~*android|iphone|windowss+(?:ce|phone) 1;
        ~*symbian|sonyericsson|samsung|lg|blackberry 1;
        ~*mobile 1;
}

map $request_uri $is_uri {
        default 1;
        ~^/community/(?:forum/|topic/|members/|feedback/|user/|$) 0;
}

map $http_cookie $is_cookie {
        default 0;
        ~member_id 1;
        ~pass_hash 1;
        ~mobilebrowser 1;
        ~guestskinchoice 1;
        ~language 1;
}

map $http_cookie $cache_uid {
        default nil;
        ~session_id=(?<session_id>[[:graph:]]+) $session_id;
}

site.conf (just the PHP part)

        location ~ .php$ {
                include conf.d/fastcgi_params_php.inc;

                fastcgi_cache ipb;
                fastcgi_cache_valid 200 2m;
                fastcgi_cache_key "$cache_uid@$scheme$request_method$host$request_uri";
                fastcgi_cache_use_stale off;
                fastcgi_no_cache $is_cookie $is_args $is_mobile $is_uri;
                fastcgi_cache_bypass $is_cookie $is_args $is_mobile $is_uri;
                fastcgi_ignore_headers Cache-Control Expires Set-Cookie;
                add_header X-Cache $upstream_cache_status;
        }

This creates correctly identifies mobile devices and uses the session token as part of the cache key - the importance of this is that without it, a guest who subsequently logs in can have their login hijacked by the next guest.

[.Nuno.]

This creates correctly identifies mobile devices and uses the session token as part of the cache key - the importance of this is that without it, a guest who subsequently logs in can have their login hijacked by the next guest.

Hi,

I agree with you, adding the session to the key is a secure measure that avoids session hijacked but defeats the cache propose, since you are caching the same page n times the number of users that visit this page ...

I would prefer a way to remove the session cookie for guests, maybe a mod!

[maddog107_merged]

anyone tried nginx cache or varnish cache on 3.4.x?

[.Nuno.]

I'm using nginx cache with 3.4

[maddog107_merged]

^ same config as before or you had to modify it a bit?