Jump to content

Unbelievable results with Varnish

mat206
mat206
in Classic self-hosted technical help

Featured Replies

Hi,

No I didn't.

I'm now testing nginx cache (fastcgi) with some great results and with less memory than with varnish.
At this point it's working like varnish but without forcing purging. I've set cache_valid to 15m.
I'll open a new topic, since this one is for varnish

  • 4 weeks later...
  • Replies 202
  • Views 8.9k
  • Created
  • Last Reply

if its difficult to overcome htaccess issues (due to how I am running ip.content) would you think varnish would work better than nginx?

  • 1 month later...
  On 6/10/2012 at 9:02 AM, .Nuno. said:

Hi,



No I didn't.



I'm now testing nginx cache (fastcgi) with some great results and with less memory than with varnish.


At this point it's working like varnish but without forcing purging. I've set cache_valid to 15m.


I'll open a new topic, since this one is for varnish




Yup post #122. :lol:
  • 2 weeks later...

Hello, im late in this thread... any one has the solution to have the client ip in nginx logs? i see varnish IP in the logs ¬¬

Find it! :)

proxy_set_header        Host            $host;

proxy_set_header        X-Real-IP       $remote_addr;

proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

set_real_ip_from        REPLACE-W-VARNISH-IP;

real_ip_header          X-Forwarded-For;



PS: this also fixes the "users online" on board index

  • 1 month later...

Quick Question, hate to add this to sunch a long thread, but how do I simply disable varnish for logged in users/members? We only want to server varnish to guests, and give our members the freshest content.

thats how it supposed to work with the config posted in this thread

What its for: Unixy Varnish WHM/CPANEL plugin

With the help of Unixy Staff I give you a default.vcl for the Unixy Varnish WHM/CPANEL plugin which can be found here: http://www.unixy.net/varnish/ (Just FYI its a paid plugin, but worth everypenny, varnish setup was a snap with it. )

What it does: Routes members/mobile users around cache.

This modification is quite simple and avoids IPBoard problem spots by passing logged-in members and mobile visitors around the varnish cache, and straight to the webserver.

The first default.vcl is for non-cloudflare UNixy Plugin users, the second includes the code necessary to get client-IP's to work correctly with cloudflare.

Enjoy!

Without Cloudflare X-Forward:

###################################################
# Copyright © UNIXY  -  http://www.unixy.net    #
# The leading truly fully managed server provider #
###################################################

include "/etc/varnish/cpanel.backend.vcl";

include "/etc/varnish/backends.vcl";

# mod_security rules
include "/etc/varnish/security.vcl";

sub vcl_recv {

# Use the default backend for all other requests
set req.backend = default;

# Setup the different backends logic
include "/etc/varnish/acllogic.vcl";

# Allow a grace period for offering "stale" data in case backend lags
set req.grace = 5m;

remove req.http.X-Forwarded-For;
set req.http.X-Forwarded-For = client.ip;

# cPanel URLs
include "/etc/varnish/cpanel.url.vcl";

# Properly handle different encoding types
if (req.http.Accept-Encoding) {
	if (req.url ~ ".(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|ico)$") {
		# No point in compressing these
		remove req.http.Accept-Encoding;
	} elsif (req.http.Accept-Encoding ~ "gzip") {
		set req.http.Accept-Encoding = "gzip";
	} elsif (req.http.Accept-Encoding ~ "deflate") {
		set req.http.Accept-Encoding = "deflate";
	} else {
		# unkown algorithm
		remove req.http.Accept-Encoding;
	}
}

# Set up disabled
include "/etc/varnish/disabled.vcl";

# Exclude upgrade, install, server-status, etc
include "/etc/varnish/known.exclude.vcl";

# Set up exceptions
include "/etc/varnish/url.exclude.vcl";

# Set up exceptions
include "/etc/varnish/debugurl.exclude.vcl";

# Set up exceptions
include "/etc/varnish/vhost.exclude.vcl";

# Set up vhost+url exceptions
include "/etc/varnish/vhosturl.exclude.vcl";

# Set up cPanel reseller exceptions
include "/etc/varnish/reseller.exclude.vcl";

# Restart rule for bfile recv
include "/etc/varnish/bigfile.recv.vcl";


if (req.request == "PURGE") {
        if (!client.ip ~ acl127_0_0_1) {error 405 "Not permitted";}
        return (lookup);
}

## Default request checks
if (req.request != "GET" &&
req.request != "HEAD" &&
req.request != "PUT" &&
req.request != "POST" &&
req.request != "TRACE" &&
req.request != "OPTIONS" &&
req.request != "DELETE") {
	return (pipe);
}

if (req.request != "GET" && req.request != "HEAD") {
	return (pass);
}

## Modified from default to allow caching if cookies are set, but not http auth
if (req.http.Authorization) {
	return (pass);
}


 ## == Mobile ==
  if (req.http.User-Agent ~ "(iPad|iPhone|iPod|Android|SymbianOS|^BlackBerry|^SonyEricsson|^Nokia|^SAMSUNG|^LG)") {
    return(pass);
  }

 if (req.url ~ "(section=markasread|section=login|register)") {
        return(pass);
}
else {

  if ((req.http.Cookie ~ "member_id=" && req.http.Cookie !~ "member_id=(0|-1)") || req.http.Cookie ~ "guestSkinChoice=")  {
    return(pass);
  } else {
    if (req.url ~ "^/forum" || req.url ~ "^/topic" || req.url ~ "^/public") {
      unset req.http.cookie;
      set req.grace = 15s;
    }
  }
}


include "/etc/varnish/versioning.static.vcl";

## Remove has_js and Google Analytics cookies.
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;s*)(__[a-z]+|has_js)=[^;]*", "");

set req.http.Cookie = regsub(req.http.Cookie, "^;s*", "");

if (req.http.Cookie ~ "^s*$") {
	unset req.http.Cookie;
}

include "/etc/varnish/slashdot.recv.vcl";

# Cache things with these extensions
if (req.url ~ ".(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|pdf)$" && ! (req.url ~ ".(php)") ) {
    unset req.http.Cookie;
    return (lookup);
}

return (lookup);
}

sub vcl_fetch {

set beresp.ttl = 40s;
set beresp.http.Server = " - Web acceleration by http://www.unixy.net/varnish ";

# Turn off Varnish gzip processing
include "/etc/varnish/gzip.off.vcl";

# Grace to allow varnish to serve content if backend is lagged
set beresp.grace = 5m;

# Restart rule bfile for fetch
include "/etc/varnish/bigfile.fetch.vcl";

# These status codes should always pass through and never cache.
if (beresp.status == 503 || beresp.status == 500) {
	set beresp.http.X-Cacheable = "NO: beresp.status";
	set beresp.http.X-Cacheable-status = beresp.status;
	return (hit_for_pass);
}

if (beresp.status == 404) {
	set beresp.http.magicmarker = "1";
	set beresp.http.X-Cacheable = "YES";
	set beresp.ttl = 20s;
	return (deliver);
}

/* Remove Expires from backend, it's not long enough */    
unset beresp.http.expires;

if (req.url ~ ".(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|pdf|ico)$" && ! (req.url ~ ".(php)") ) {
	unset beresp.http.set-cookie;
	include "/etc/varnish/static.ttl.vcl";
}
include "/etc/varnish/slashdot.fetch.vcl"; 
else {
	include "/etc/varnish/dynamic.ttl.vcl";
}

/* marker for vcl_deliver to reset Age: */
set beresp.http.magicmarker = "1";

# All tests passed, therefore item is cacheable
set beresp.http.X-Cacheable = "YES";

return (deliver);
}

sub vcl_deliver {

  # From http://varnish-cache.org/wiki/VCLExampleLongerCaching
  if (resp.http.magicmarker) {
     /* Remove the magic marker */
     unset resp.http.magicmarker;

     /* By definition we have a fresh object */
     set resp.http.age = "0";
   }

   #add cache hit data
   if (obj.hits > 0) {
     #if hit add hit count
     set resp.http.X-Cache = "HIT";
     set resp.http.X-Cache-Hits = obj.hits;
   }
else {
     set resp.http.X-Cache = "MISS";
   }

}

sub vcl_error {

if (obj.status == 503 && req.restarts < 5) {
set obj.http.X-Restarts = req.restarts;
return (restart);
}

}

# Added to let users force refresh
sub vcl_hit {

if (obj.ttl < 1s) {
	return (pass);
}

if (req.http.Cache-Control ~ "no-cache") {
# Ignore requests via proxy caches,  IE users and badly behaved crawlers
# like msnbot that send no-cache with every request.
if (! (req.http.Via || req.http.User-Agent ~ "bot|MSIE|HostTracker")) {
	set obj.ttl = 0s;
	return (restart);
} 
}

return (deliver);

}

sub vcl_hash {

	hash_data(req.http.cookie);
}

With Cloudflare X-Forward Running:

###################################################
# Copyright © UNIXY  -  http://www.unixy.net    #
# The leading truly fully managed server provider #
###################################################

include "/etc/varnish/cpanel.backend.vcl";

include "/etc/varnish/backends.vcl";

# mod_security rules
include "/etc/varnish/security.vcl";

sub vcl_recv {

# Use the default backend for all other requests
set req.backend = default;

# Setup the different backends logic
include "/etc/varnish/acllogic.vcl";

# Allow a grace period for offering "stale" data in case backend lags
set req.grace = 5m;

#cloudflare
  remove req.http.X-Forwarded-For;
  if (req.http.cf-connecting-ip) {
    set req.http.X-Forwarded-For = req.http.cf-connecting-ip;
  } else {
    set req.http.X-Forwarded-For = client.ip;
  }

# cPanel URLs
include "/etc/varnish/cpanel.url.vcl";

# Properly handle different encoding types
if (req.http.Accept-Encoding) {
	if (req.url ~ ".(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|ico)$") {
		# No point in compressing these
		remove req.http.Accept-Encoding;
	} elsif (req.http.Accept-Encoding ~ "gzip") {
		set req.http.Accept-Encoding = "gzip";
	} elsif (req.http.Accept-Encoding ~ "deflate") {
		set req.http.Accept-Encoding = "deflate";
	} else {
		# unkown algorithm
		remove req.http.Accept-Encoding;
	}
}

# Set up disabled
include "/etc/varnish/disabled.vcl";

# Exclude upgrade, install, server-status, etc
include "/etc/varnish/known.exclude.vcl";

# Set up exceptions
include "/etc/varnish/url.exclude.vcl";

# Set up exceptions
include "/etc/varnish/debugurl.exclude.vcl";

# Set up exceptions
include "/etc/varnish/vhost.exclude.vcl";

# Set up vhost+url exceptions
include "/etc/varnish/vhosturl.exclude.vcl";

# Set up cPanel reseller exceptions
include "/etc/varnish/reseller.exclude.vcl";

# Restart rule for bfile recv
include "/etc/varnish/bigfile.recv.vcl";


if (req.request == "PURGE") {
        if (!client.ip ~ acl127_0_0_1) {error 405 "Not permitted";}
        return (lookup);
}

## Default request checks
if (req.request != "GET" &&
req.request != "HEAD" &&
req.request != "PUT" &&
req.request != "POST" &&
req.request != "TRACE" &&
req.request != "OPTIONS" &&
req.request != "DELETE") {
	return (pipe);
}

if (req.request != "GET" && req.request != "HEAD") {
	return (pass);
}

## Modified from default to allow caching if cookies are set, but not http auth
if (req.http.Authorization) {
	return (pass);
}


 ## == Mobile ==
  if (req.http.User-Agent ~ "(iPad|iPhone|iPod|Android|SymbianOS|^BlackBerry|^SonyEricsson|^Nokia|^SAMSUNG|^LG)") {
    return(pass);
  }

 if (req.url ~ "(section=markasread|section=login|register)") {
        return(pass);
}
else {

  if ((req.http.Cookie ~ "member_id=" && req.http.Cookie !~ "member_id=(0|-1)") || req.http.Cookie ~ "guestSkinChoice=")  {
    return(pass);
  } else {
    if (req.url ~ "^/forum" || req.url ~ "^/topic" || req.url ~ "^/public") {
      unset req.http.cookie;
      set req.grace = 15s;
    }
  }
}


include "/etc/varnish/versioning.static.vcl";

## Remove has_js and Google Analytics cookies.
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;s*)(__[a-z]+|has_js)=[^;]*", "");

set req.http.Cookie = regsub(req.http.Cookie, "^;s*", "");

if (req.http.Cookie ~ "^s*$") {
	unset req.http.Cookie;
}

include "/etc/varnish/slashdot.recv.vcl";

# Cache things with these extensions
if (req.url ~ ".(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|pdf)$" && ! (req.url ~ ".(php)") ) {
    unset req.http.Cookie;
    return (lookup);
}

return (lookup);
}

sub vcl_fetch {

set beresp.ttl = 40s;
set beresp.http.Server = " - Web acceleration by http://www.unixy.net/varnish ";

# Turn off Varnish gzip processing
include "/etc/varnish/gzip.off.vcl";

# Grace to allow varnish to serve content if backend is lagged
set beresp.grace = 5m;

# Restart rule bfile for fetch
include "/etc/varnish/bigfile.fetch.vcl";

# These status codes should always pass through and never cache.
if (beresp.status == 503 || beresp.status == 500) {
	set beresp.http.X-Cacheable = "NO: beresp.status";
	set beresp.http.X-Cacheable-status = beresp.status;
	return (hit_for_pass);
}

if (beresp.status == 404) {
	set beresp.http.magicmarker = "1";
	set beresp.http.X-Cacheable = "YES";
	set beresp.ttl = 20s;
	return (deliver);
}

/* Remove Expires from backend, it's not long enough */    
unset beresp.http.expires;

if (req.url ~ ".(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|pdf|ico)$" && ! (req.url ~ ".(php)") ) {
	unset beresp.http.set-cookie;
	include "/etc/varnish/static.ttl.vcl";
}
include "/etc/varnish/slashdot.fetch.vcl"; 
else {
	include "/etc/varnish/dynamic.ttl.vcl";
}

/* marker for vcl_deliver to reset Age: */
set beresp.http.magicmarker = "1";

# All tests passed, therefore item is cacheable
set beresp.http.X-Cacheable = "YES";

return (deliver);
}

sub vcl_deliver {

  # From http://varnish-cache.org/wiki/VCLExampleLongerCaching
  if (resp.http.magicmarker) {
     /* Remove the magic marker */
     unset resp.http.magicmarker;

     /* By definition we have a fresh object */
     set resp.http.age = "0";
   }

   #add cache hit data
   if (obj.hits > 0) {
     #if hit add hit count
     set resp.http.X-Cache = "HIT";
     set resp.http.X-Cache-Hits = obj.hits;
   }
else {
     set resp.http.X-Cache = "MISS";
   }

}

sub vcl_error {

if (obj.status == 503 && req.restarts < 5) {
set obj.http.X-Restarts = req.restarts;
return (restart);
}

}

# Added to let users force refresh
sub vcl_hit {

if (obj.ttl < 1s) {
	return (pass);
}

if (req.http.Cache-Control ~ "no-cache") {
# Ignore requests via proxy caches,  IE users and badly behaved crawlers
# like msnbot that send no-cache with every request.
if (! (req.http.Via || req.http.User-Agent ~ "bot|MSIE|HostTracker")) {
	set obj.ttl = 0s;
	return (restart);
} 
}

return (deliver);

}

sub vcl_hash {

	hash_data(req.http.cookie);
}

so this basically stops the cookie settings the org unixy one uses?

they did not believe me when I said the plugin logs IP members out when FURLS used.

if NOT using furls the plugin works fine stock.

  On 10/17/2012 at 9:38 PM, Dmacleo said:

so this basically stops the cookie settings the org unixy one uses?

they did not believe me when I said the plugin logs IP members out when FURLS used.

if NOT using furls the plugin works fine stock.

Odd I never ran into a logging out issue. We had more issues along the lines of posts not deleting, double posts being created, lots of minor issues. This fixed 100% of them.

But I'd say give it a shot if your running the UNixy plugin, other than the changes I mentioned for mobile and registered members, its absolutely stock.

We've been running this config for around 8 hours now and not a single issue. The nicest thing about it is giving members priority, by passing them straight to the server is they NEVER get stale content, and we can now give guests slightly older content, I bumped it to 1200 second ttl, might put it back down to 900 but I don't think another 5 minutes will make a huge difference for guests. And it only increases our cache hit rate.

  On 10/17/2012 at 9:38 PM, Dmacleo said:

so this basically stops the cookie settings the org unixy one uses?

they did not believe me when I said the plugin logs IP members out when FURLS used.

if NOT using furls the plugin works fine stock.

On another note you might want to be careful as the the VCL's appear very different between directadmin and whm

just send them a support message and say you want to integrate this code and they should do it for you no problem.

 ## == Mobile ==
  if (req.http.User-Agent ~ "(iPad|iPhone|iPod|Android|SymbianOS|^BlackBerry|^SonyEricsson|^Nokia|^SAMSUNG|^LG)") {
    return(pass);
  }

 if (req.url ~ "(section=markasread|section=login|register)") {
        return(pass);
}
else {

  if ((req.http.Cookie ~ "member_id=" && req.http.Cookie !~ "member_id=(0|-1)") || req.http.Cookie ~ "guestSkinChoice=")  {
    return(pass);
  } else {
    if (req.url ~ "^/forum" || req.url ~ "^/topic" || req.url ~ "^/public") {
      unset req.http.cookie;
      set req.grace = 15s;
    }
  }
}

are you running ip.content as home page? been trying to nail it down to see if its caused by the rewrites used when content is homepage since with furls off its flawless.

with furls off it worked well for logged in/dynamic stuff for me.

I have been messing with it for month or so to see if I can narrow it down, tried it using htaccess or rewrite rules in httpd include editor and no htaccess.

I am tempted to not run the furls as it really helped spikes., using cpnginx now and that works with furls but does not handle spikes as well.

I am on cpanel/whm now.

just got tired of directadmin.

  On 10/18/2012 at 12:50 AM, Dmacleo said:

are you running ip.content as home page? been trying to nail it down to see if its caused by the rewrites used when content is homepage since with furls off its flawless.

with furls off it worked well for logged in/dynamic stuff for me.

I have been messing with it for month or so to see if I can narrow it down, tried it using htaccess or rewrite rules in httpd include editor and no htaccess.

I am tempted to not run the furls as it really helped spikes., using cpnginx now and that works with furls but does not handle spikes as well.

nah we dont use ipcontent, just straight IPBoard, and were adding IP.Downloads in the near future to our site. That sounds like an odd issues though.

I would honestly say hit up gary hes been able to dig me out of some holes when I've landed in them: http://community.invisionpower.com/user/137679-gary/

Generally works for donations, but has been incredibly helpful whenever I've run into tough issues. He is also amazing at helping get a stable, speedy, and secure, VPS/Dedi running.

yeah on test board w/o content it was fine no matter what we did.

but since test board is on same server as live I was not sure if that was fluke.

you running straight board also w/ no issues leads me to believe its how content and board fight for the index file :) :)

this is actually good news, helps firm up my thinking on it.

can run w/ this vcl for guests and when ipb 4 comes out where everything isn't a subset of forum application I think issue will self resolve.

  On 10/18/2012 at 1:16 AM, Dmacleo said:

yeah on test board w/o content it was fine no matter what we did.

but since test board is on same server as live I was not sure if that was fluke.

you running straight board also w/ no issues leads me to believe its how content and board fight for the index file smile.pngsmile.png

this is actually good news, helps firm up my thinking on it.

can run w/ this vcl for guests and when ipb 4 comes out where everything isn't a subset of forum application I think issue will self resolve.

Awesome glad it helped :)

Yeah I can't wait for IPBoard 4, InvisionPower has really been on a good track of improvements since IPBoard 3.0 launched, they are really leaving VB in the dust IMO.

just need to get off butt, remove cpnginx and reinstall this and add this to vcl.

just have not taken the 10 minutes to do it LOL

  • 3 weeks later...
  • Author

Here is my current VCL configuration for those interested:

"> <html> <head> <title>"} + obj.status + " " + obj.response + {"</title> </head> <body> <h1>Error "} + obj.status + " " + obj.response + {"</h1> <p>"} + obj.response + {"</p> <h3>Guru Meditation:</h3> <p>XID: "} + req.xid + {"</p> <address> <a href="">Nostalgia</a><br/> </address> </body> </html> "}; } return (deliver); } 

# Current working server

backend default {

.host = "192.168.1.1";

.port = "8081";

}



sub vcl_fetch {


        ## Remove the X-Forwarded-For header if it exists.

        remove req.http.X-Forwarded-For;


        ## insert the client IP address as X-Forwarded-For. This is the normal IP address of the user.

        set req.http.X-Forwarded-For = client.ip;


    // Dont cache 302 redirects and anything else other than what should be cached

    if (beresp.status != 200 && beresp.status != 404) {

        set beresp.ttl = 0s;

        return (deliver);

    }


    if (req.url ~ ".(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|pdf|ico)$" && !(req.url ~ ".(php)") )

    {

        unset beresp.http.set-cookie;

        set beresp.ttl = 600s;

    }


      unset beresp.http.Server;

      set beresp.http.Server = "GDNET";



      ## Deliver the content

      return(deliver);

}


## Deliver

sub vcl_deliver {



        ## We'll be hiding some headers added by Varnish. We want to make sure people are not seeing we're using Varnish.

        ## Since we're not caching (yet), why bother telling people we use it?

        remove resp.http.X-Varnish;

        remove resp.http.Via;

        remove resp.http.Age;


        ## We'd like to hide the X-Powered-By headers. Nobody has to know we can run PHP and have version xyz of it.

        remove resp.http.X-Powered-By;


        if (obj.hits > 0) {

                set resp.http.X-Cache = "HIT";

        set resp.http.X-Cache-Hits = obj.hits;


        } else {

                set resp.http.X-Cache = "MISS";

        }


        set resp.http.X-Success-Kid = "Linked from Reddit.. Didnt Crash"; #  You can probably remove this


}


sub vcl_recv {


    set req.backend = default;


    remove req.http.X-Forwarded-For;

    set    req.http.X-Forwarded-For = client.ip; //req.http.rlnclientipaddr;



    if (req.request != "GET" && req.request != "HEAD")

    {

        return (pipe);

    }


    // Can't afford timeouts

    if (req.url ~ "files" || req.url ~ "ajax")

    {

             return (pipe);

    }


    ## Modified from default to allow caching if cookies are set, but not http auth

    if (req.http.Authorization)

    {

        return (pass);

    }



    set req.http.X-Device = "pc";

    if (    req.http.User-Agent ~ "iPad" || req.http.User-Agent ~ "iP(hone|od)" ||

        req.http.User-Agent ~ "Android" || req.http.User-Agent ~ "SymbianOS" ||

        req.http.User-Agent ~ "^BlackBerry" || req.http.User-Agent ~ "^SonyEricsson" ||

        req.http.User-Agent ~ "^Nokia" || req.http.User-Agent ~ "^SAMSUNG" || req.http.User-Agent ~ "^LG")

    {

        set req.http.X-Device = "mobile";

    }



    if (req.url ~ ".(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|pdf|ico)$")

    {

        unset req.http.cookie;

        return(lookup);

    }


    if (req.url ~ "(section=markasread|section=login|register)")

    {

        return (pass);

    }

    else

    {

        // Remove has_js and Google Analytics __* cookies.

        set req.http.Cookie = regsuball(req.http.Cookie, "(^|;s*)(_[_a-z]+|has_js)=[^;]*", "");


        // Remove a ";" prefix, if present.

        set req.http.Cookie = regsub(req.http.Cookie, "^;s*", "");


        if (req.http.cookie && (req.http.cookie ~ "member_id" && req.http.cookie !~ "member_id=(0|-1)"))

        {

            return (pass);

        }

        else

        {

            unset req.http.cookie;

              set req.grace = 15s;

        }


    }


}


sub vcl_hash {


    hash_data(req.http.host);

    hash_data(req.url);

    hash_data(req.http.X-Device);


    return (hash);

}



sub vcl_error {

set obj.http.Content-Type = "text/html; charset=utf-8";


if ( obj.status >= 500 && obj.status <= 505) {

synthetic {"

<html>

<head>

<title>Error 503</title>

</head>

<body>

Our server is temporarily down - Please check back in 15 minutes.

<br/><br/>

</body>

</html>

"};

} else {

synthetic {"

<?xml version="1.0" encoding="utf-8"?>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"

"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtdhttp://archive.org/

are you using ip content as home page and do you get users logged out when rewrites being used for furls?

  • Author
  On 11/5/2012 at 8:06 PM, Dmacleo said:

are you using ip content as home page and do you get users logged out when rewrites being used for furls?

I use IP.Content as a home page and don't suffer from that problem as far as I can tell.

iirc your using no cpanel or any panel right?

used unixy varnish plugin on cpanel/directadmin on my end which DOES cache for logged in users too.

issue I had was with the rewrites for furls used users got logged out, turn the use htaccess off in acp and, with less uglier urls, it worked perfect.

on test board w/o content as home page (so no constants.php or anything) did not have the issues.

use a vcl that bypasses cookies/login and its all good too.

think at some point I will take a few moments and try your vcl.

  • Author

I'm using Varnish 3.0+ btw. No Cpanel. There are some VCL language differences (like hash_data() in vcl_hash) that might need tweaking if you use pre-3.0. Here's an updated version that will also temporarily cache content for logged on users.

UPDATED VCL TO ALSO TEMPORARILY CACHE LOGGED IN USERS CONTENT FOR THEM (caches their content for 20 seconds in the event they keep refreshing the page)

"> <html> <head> <title>"} + obj.status + " " + obj.response + {"</title> </head> <body> <h1>Error "} + obj.status + " " + obj.response + {"</h1> <p>"} + obj.response + {"</p> <h3>Guru Meditation:</h3> <p>XID: "} + req.xid + {"</p> <address> <a href="">Nostalgia</a><br/> </address> </body> </html> "}; } return (deliver); } 

# Current working server

backend default {

.host = "192.168.1.8";

.port = "8081";

}



sub vcl_fetch {


        ## Remove the X-Forwarded-For header if it exists.

        remove req.http.X-Forwarded-For;

        ## insert the client IP address as X-Forwarded-For. This is the normal IP address of the user.

        //set req.http.X-Forwarded-For = req.http.rlnclientipaddr;

        set req.http.X-Forwarded-For = client.ip;


        // Dont cache 302 redirects and anything else other than what should be cached

        if (beresp.status != 200 && beresp.status != 404) {

        set beresp.ttl = 0s;

        return (deliver);

        }


        // Added 11/5/12

        if (req.url ~ ".(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|pdf|ico)$" && !

        (req.url ~ ".(php)") )

        {

        unset beresp.http.set-cookie;

        set beresp.ttl = 600s;

        }



        else if (req.http.cookie && (req.http.cookie ~ "member_id" && req.http.cookie !~ "member_id=(0|-1)"))

        {

        set beresp.ttl = 20s;

        set beresp.grace = 15s;

        }

        else

        {

        set beresp.ttl = 60s;

        set beresp.grace = 15s;

        }



        unset beresp.http.Server;

        set beresp.http.Server = "IPS";



        ## Deliver the content

        return(deliver);

}


## Deliver

sub vcl_deliver {


        ## We'll be hiding some headers added by Varnish. We want to make sure people are not seeing we're using Varnish.

        ## Since we're not caching (yet), why bother telling people we use it?

        remove resp.http.X-Varnish;

        remove resp.http.Via;

        remove resp.http.Age;


        ## We'd like to hide the X-Powered-By headers. Nobody has to know we can run PHP and have version xyz of it.

        remove resp.http.X-Powered-By;


        if (obj.hits > 0) {

                set resp.http.X-Cache = "HIT";

                set resp.http.X-Cache-Hits = obj.hits;


        } else {

                set resp.http.X-Cache = "MISS";

        }



        #set resp.http.X-Rick-Would-Never = "Let you down";  #  You can probably remove this

        set resp.http.X-Success-Kid = "Linked from Reddit.. Didnt Crash";


}


sub vcl_recv {


        set req.backend = default;


        remove req.http.X-Forwarded-For;

        set    req.http.X-Forwarded-For = client.ip; //req.http.rlnclientipaddr;



        set req.grace = 15s;


        if (req.request != "GET" && req.request != "HEAD")

        {

          return (pipe);

        }


        // Can't afford timeouts

        if (req.url ~ "files" || req.url ~ "ajax")

        {

              return (pipe);

        }


        ## Modified from default to allow caching if cookies are set, but not http auth

        if (req.http.Authorization)

        {

        return (pass);

        }


        # IF THINGS DONT WORK, UNCOMMENT THIS NEXT LINE

        #return (pass);






        set req.http.X-Device = "pc";

        if (req.http.User-Agent ~ "iPad" || req.http.User-Agent ~ "iP(hone|od)" ||

        req.http.User-Agent ~ "Android" || req.http.User-Agent ~ "SymbianOS" || req.http.User-Agent ~ "^BlackBerry" || req.http.User-Agent ~ "^SonyEricsson" ||

        req.http.User-Agent ~ "^Nokia" || req.http.User-Agent ~ "^SAMSUNG" || req.http.User-Agent ~ "^LG")

        {

        set req.http.X-Device = "mobile";

        }



        if (req.url ~ ".(js|css|jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|pdf|ico)$")

        {

          unset req.http.cookie;

          return(lookup);

        }


        if (req.url ~ "(section=markasread|section=login|register)")

        {

          return (pass);

        }

        else

        {

           // Remove has_js and Google Analytics __* cookies.

           set req.http.Cookie = regsuball(req.http.Cookie, "(^|;s*)(_[_a-z]+|has_js)=[^;]*", "");


           // Remove a ";" prefix, if present.

           set req.http.Cookie = regsub(req.http.Cookie, "^;s*", "");


          if (req.http.cookie && (req.http.cookie ~ "member_id" && req.http.cookie !~

          "member_id=(0|-1)"))

          {

            return (lookup);  #return (pass);

          }

          else

          {

            unset req.http.cookie;

          }


        }


}


sub vcl_hash {


        if (req.http.Cookie && req.http.Cookie ~ "member_id")

        {

            hash_data(regsub( req.http.Cookie, "^.*?member_id=([^;]*);*.*$", "1"));

        }


        hash_data(req.http.host);

        hash_data(req.url);

        hash_data(req.http.X-Device);


        return (hash);

}



sub vcl_error {

set obj.http.Content-Type = "text/html; charset=utf-8";


if ( obj.status >= 500 && obj.status <= 505) {

synthetic {"

<html>

<head>

<title>Error 503</title>

</head>

<body>

Our server is temporarily down - Please check back in 15 minutes.

<br/><br/>

</body>

</html>

"};

} else {

synthetic {"

<?xml version="1.0" encoding="utf-8"?>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"

"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtdhttp://archive.org/

the plugin d/ls the latest varnish upon each install, I forget specific version now but know it started with 3

will give this a try later this week, thanks.

  • 2 months later...

I've tried nearly all VCL configs here, on 3.4.2

but none of them seems to cache anything else than "index.php?blablah.css" files (all other php files are a MISS)

Especially, the above configs do not cache forums, threads, etc on my stock 3.4 install, for a not logged in user.

I'm guessing there is a new cookie in 3.4 that creates a problem. I'm new in varnish and I'd appreciate your thought on this issue...

Archived

This topic is now archived and is closed to further replies.

Go to topic listing

Recently Browsing 0

  • No registered users viewing this page.