Unbelievable results with Varnish

[blair]

Have you done any before/after tests at webpagetest.org?

[Amy T]

Have you done any before/after tests at webpagetest.org?

I did not do any tests there but I did an after and before on Pingdoms page load tests.
I got 16seconds before and 9 after.

[blair]

Nice. :thumbsup:

[.Nuno.]

Hello,

There is a small bug in the code above because "copy & paste"

The "Normalise Accept-Encoding" should be:

  ## == Normalise Accept-Encoding ==

  if (req.http.Accept-Encoding) {

      if (req.url ~ ".(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {

          # No point in compressing these

          remove req.http.Accept-Encoding;

      } elsif (req.http.Accept-Encoding ~ "gzip") {

          set req.http.Accept-Encoding = "gzip";

      } elsif (req.http.Accept-Encoding ~ "deflate") {

          set req.http.Accept-Encoding = "deflate";

      } else {

          # unkown algorithm

          remove req.http.Accept-Encoding;

      }

  }

[Luis Manson]

im not sure if i made any mistakes, but i always have a MISS on the headers, unlest i request the same url again...

i should receive a hit on index at least, right?

[.Nuno.]

im not sure if i made any mistakes, but i always have a MISS on the headers, unlest i request the same url again...

i should receive a hit on index at least, right?

You first hit will always be a MISS, only the subsequent hits will give you an HIT (if they occur before they expire).

[Luis Manson]

thanks

[Luis Manson]

im testing this, but just enabling varnish i lost all online users from >400 to ~2/3 online users

i also, with no varnish have a modtids and session_id cookies, is this ok?

[.Nuno.]

im testing this, but just enabling varnish i lost all online users from >400 to ~2/3 online users

i also, with no varnish have a modtids and session_id cookies, is this ok?

I don't know your setup, but has I said, mine is nginx -> varnish -> apache2+mod_php plus ESI to count topics/images views.
I have not tested another setup.
Can you give us more information?

[Luis Manson]

sorry. varnish -> apache on 8018 -> mod_php + xcache
im using your config from post 67

im missing ESI, what did you do beside editing apache/varnish/nxigx settings? (file creatin/template edits?)

also if u dont care could you post your apache/nginx settings?

[.Nuno.]

sorry. varnish -> apache on 8018 -> mod_php + xcache

im using your config from post 67

im missing ESI, what did you do beside editing apache/varnish/nxigx settings? (file creatin/template edits?)

also if u dont care could you post your apache/nginx settings?

Hi,

You can remove the ESI part of the setup.
Here is an exemple of a setup you can start with.


/etc/nginx/conf.d/proxy.conf

proxy_redirect 	off;


proxy_set_header   Host 			$host;

proxy_set_header   X-Real-IP        $remote_addr;

proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;


client_max_body_size   	10m;

client_body_buffer_size    128k;


proxy_connect_timeout      90;

proxy_send_timeout 		90;

proxy_read_timeout 		90;


proxy_buffer_size          32k;

proxy_buffers              4 32k;

proxy_busy_buffers_size    64k;

proxy_temp_file_write_size 64k;

/etc/nginx/sites-available/example.com.conf

server {

        listen   80;

        server_name www.example.com;

        access_log off;

        root    /srv/www/example.com/public;


     	# Main location

        location / {

            access_log  /var/log/nginx/example-access.log  main;

            proxy_pass  http://127.0.0.1:8080/;

        }


        # Static files location

        location ~* ^.+.(jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js)$ {

            access_log off;

            expires 30d;

        	}

	}

/etc/apache2/sites-available/example.com.conf

<VirtualHost *:8080>


  # Admin email, Server Name (domain name) and any aliases

  ServerAdmin mail@example.com

  ServerName  example.com

  ServerAlias www.example.com

  php_value mail.force_extra_parameters "-f mail@example.com"



  DirectoryIndex index.php index.html

  DocumentRoot /srv/www/example.com/public


  # Custom log file locations

  LogLevel warn

  ErrorLog  /var/log/apache2/example.com-error.log

  CustomLog /var/log/apache2/example.com-access.log combined env=!no-log


  # RPAF proxy

  RPAFenable On

  RPAFsethostname On

  RPAFproxy_ips 127.0.0.1


  <Directory /srv/www/example.com/public>

    Options -MultiViews -Indexes

    AllowOverride All

    Order allow,deny

    Allow from all

  </Directory>


</VirtualHost>

[Luis Manson]

thanks! i will try this this week, about esi i think i will use it, for what i know is to keep user views/counts working

[.Nuno.]

Instructions to use ESI are here.
Good Luck.

[Luis Manson]

thanks again, im missing another bit if you use nginx - varnish - apache
and in your previous post apache listens on 8080 but nginx proxies to that post i dont see where varnish fits, unless its localhost, but wont that make apache complain about localhost:8080 ?

[.Nuno.]

thanks again, im missing another bit if you use nginx - varnish - apache

and in your previous post apache listens on 8080 but nginx proxies to that post i dont see where varnish fits, unless its localhost, but wont that make apache complain about localhost:8080 ?

I thought you just want nginx -> apache...

edit your nginx.conf and add to your http settings:

    upstream varnish  {

      server 127.0.0.1:6081;

      server 127.0.0.1:8080 backup;

    }

Edit your /etc/nginx/sites-available/example.com.conf and find:

proxy_pass  http://127.0.0.1:8080/;

replace with:

proxy_pass  http://varnish;

nginx listen in port 80 and proxy to varnish in port 6081 that proxies to apache in port 8080.
Note that I use Ubuntu Server and varnish setup is only for one site, so I don't check the header: http://stackoverflow...es-with-varnish

[Luis Manson]

thanks again, i got it running, seems to be working OK :)

[.Nuno.]

Keep an eye in your Site Performance @ WMT ;)

[Luis Manson]

whats wmt? :|
im using varnishhist / varnishlog

[.Nuno.]

whats wmt? :|

im using varnishhist / varnishlog

I was referring to Google Webmaster Tools Site Performance so you can see the load time evolution from now.

[Ken S.]

Hello.

I have a server that runs SEVERAL sites. I want them ALL to use Varnish. They're setup using a single IP and Name Virtual Hosts in Apache.
We have plenty of customizations and additional applications, AdSense, AdManager, etc.

I was able to get the sites showing the correct web sites, but now, some JavaScript isn't being parsed by web browsers, and instead is showing up as text when going to say, example.com or www.example.com and the rest of the site isn't showing.

Do you guys have a fix for this?

Thanks.

[altenerg]

All I used in default.vcl is the below code...

backend apache {

	    .host = "127.0.0.1";

	    .port = "8080";

}

sub vcl_fetch {

	    remove req.http.X-Forwarded-For;

	    set    req.http.X-Forwarded-For = req.http.rlnclientipaddr;

	    return(deliver);

}

sub vcl_deliver {

	    remove resp.http.X-Varnish;

	    remove resp.http.Via;

	    remove resp.http.Age;

	    remove resp.http.X-Powered-By;

}

Yet, I have no login issues and same performance (slightly better since index is cached) as when I try your longer vcl config. What am I missing? Why exclude index.php when its most visited on many forums?

[mat206]

Last Sunday we had an instance of about 16,000 visitors over a 30 minute time span. If it wasn't for Varnish our site would have been toast.. we've survived massive reddit spikes in traffic as well when we hit the front page.

As for altenberg, your configuration isn't going to cache much of anything outside of static image files. Every time the client sends cookies along with the request, Varnish is going to direct the request to the backend ApacheNginx server. The idea of most of these configs is to detect if the member_id cookie is set or is zero, so you can strip away the cookies off of the request. That way varnish treats it like a cookieless request and will use it's cache every time.

Believe me, every line of these configurations is there for a reason.

[altenerg]

Images uploading is broken. Can't attach anything. Is there a way aroundthat?

[altenerg]

Last Sunday we had an instance of about 16,000 visitors over a 30 minute time span. If it wasn't for Varnish our site would have been toast.. we've survived massive reddit spikes in traffic as well when we hit the front page. As for altenberg, your configuration isn't going to cache much of anything outside of static image files. Every time the client sends cookies along with the request, Varnish is going to direct the request to the backend ApacheNginx server. The idea of most of these configs is to detect if the member_id cookie is set or is zero, so you can strip away the cookies off of the request. That way varnish treats it like a cookieless request and will use it's cache every time. Believe me, every line of these configurations is there for a reason.

And I need to upload files as well to forum root?

[altenerg]

Figured out how to fix broken uploads. Disregard that post.

Only thing left is to figure out what files I need to upload to match with the vcl conf.

Wish there was everything in "one post" too bad IPS does not allow us to edit old posts or im sure the author would have updated the 1st post.
Maybe you can post full instructs all-in-one post in this thread or new how-to thread?