PageSpeed Web Server Module

[Makoto]

gzip static is actually just a module that lets nginx load pre-gzipped static resources :tongue: it's different from just "gzip".

It negates having to compress requested static content on-the-fly. (But like I said above, the overhead with standard gzip compression is very low anyways)

[Aussie Cable]

Can you run this on your server?

curl -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} n" http://inserturl.here

I just tested my site, with some hooks disabled (yes some hooks can cause high TTFB):

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 28342    0 28342    0     0   484k      0 --:--:-- --:--:-- --:--:--  588k
Connect: 0.010 TTFB: 0.054 Total time: 0.057

Not too shabby if I do say so myself.

If I enable some of the hooks that are currently causing me grief:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 28374    0 28374    0     0  33311      0 --:--:-- --:--:-- --:--:-- 34943
Connect: 0.040 TTFB: 0.847 Total time: 0.852

Yeah, says alot :thumbsup:

[CheersnGears]

Which hooks are giving you grief?

[CheersnGears]

Here's mine from my server:

curl -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} n" http://www.cheersandgears.com

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 87776 0 87776 0 0 133k 0 --:--:-- --:--:-- --:--:-- 148k
Connect: 0.063 TTFB: 0.637 Total time: 0.642

But shouldn't I be testing it from another machine?

[Aussie Cable]

Which hooks are giving you grief?

Id prefer not to say that publiclly, until I get a reply from the developer of the hook/s, sorry :sad:

You could disable all your hooks, turn them on one by one (testing TTFB each time you enable one), and you will soon see which ones are giving your TTFB a massive hit.

EDIT - TTFB is really not much of an issue, however it will show you what is slowing your backend down. If you optimize this, you should be all good.

[Aussie Cable]

Here's mine from my server:

But shouldn't I be testing it from another machine?

It shouldnt matter, TTFB is the same regardless of what machine you test it from.

[CheersnGears]

It shouldnt matter, TTFB is the same regardless of what machine you test it from.

That doesn't seem to be the case for me.

Id prefer not to say that publiclly, until I get a reply from the developer of the hook/s, sorry :sad:

You could disable all your hooks, turn them on one by one (testing TTFB each time you enable one), and you will soon see which ones are giving your TTFB a massive hit.

EDIT - TTFB is really not much of an issue, however it will show you what is slowing your backend down. If you optimize this, you should be all good.

I accept PMs ;-)

[Aussie Cable]

That doesn't seem to be the case for me.

I accept PMs ;-)

It should be the around the same time, regardless of machine.

For instance, when I have a high TTFB, my machine shows around 0.800 - 0.900, and from webpagetest:

Two separate machines, around the same time.

I will send you a PM :thumbsup:

[Makoto]

TTFB will not be the same no matter which machine you test from. The entire purpose of testing on the local machine is to see how long it takes to actually generate your web pages before sending the output to the client and ensuring the issue is not network related. On a good network and when your testing location is close to the server in question, it will not be that much different, but it will of course still be higher.

0.6 seconds is a rather high TTFB value. This is mine (on my production forum), for my IP.Content home page,

Connect: 0.003 TTFB: 0.064 Total time: 0.064

And for my board index,

Connect: 0.003 TTFB: 0.184 Total time: 0.184

(The average is around 100-190ms)

The average page generation time (the time it takes to request, receive, and load an entire web page) for my visitors is ~0.57 seconds, and it's around ~0.43 for US visitors only. With a TTFB of ~0.6 seconds, that means your users have to make a request for the page, wait 0.6 seconds just for the response to be sent back, then finally load the actual page itself.

I would say a good target value here is under 0.3 seconds.

You should investigate what the cause of your high value is. Hooks can of course cause problems. Bad queries or bad code, no matter where it's from, will cause you problems. It's also generally more likely for a badly programmed hook to cause you performance issues than code written by IPS :tongue:

It can also mean you need to work on tuning your server configuration, eg. MySQL.

[Aussie Cable]

TTFB will not be the same no matter which machine you test from. The entire purpose of testing on the local machine is to see how long it takes to actually generate your web pages before sending the output to the client and ensuring the issue is not network related. On a good network and when your testing location is close to the server in question, it will not be that much different, but it will of course still be higher.

Ok, so there you go, I would say that I have been corrected :lol:

I have only said that, because I have seen around the same time, on different machines, on different networks, in the past. However I am happy to accept that I stand corrected, if the information is wrong :thumbsup:

EDIT - Kirito, I would love to see a step by step thread (if you ever can be bothered) with a full server setup and config, that you have on your machine.

I would think that this information would be invaluable. Plus I would love to get those types of times on my own server.

[CheersnGears]

this is what I'm getting from my home linux machine:

curl -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} n" http://www.cheersandgears.com
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 87929 0 87929 0 0 92570 0 --:--:-- --:--:-- --:--:-- 92556
Connect: 0.182 TTFB: 0.832 Total time: 0.950

[CheersnGears]

And this is too a nearly blank html file that I put up just for the test, so no MySQL overhead to worry about.

Remote:

curl -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} n" www.cheersandgears.com/ttfb.html
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 90 100 90 0 0 250 0 --:--:-- --:--:-- --:--:-- 251
Connect: 0.304 TTFB: 0.359 Total time: 0.359

Local:

curl -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} n" www.cheersandgears.com/ttfb.html
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 90 0 90 0 0 454 0 --:--:-- --:--:-- --:--:-- 3461
Connect: 0.171 TTFB: 0.198 Total time: 0.198

Edit: After running it a few times, local has dropped a lot

curl -o /dev/null -w "Connect: %{time_connect} TTFB: %{time_starttransfer} Total time: %{time_total} n" www.cheersandgears.com/ttfb.html
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 90 0 90 0 0 2583 0 --:--:-- --:--:-- --:--:-- 45000
Connect: 0.032 TTFB: 0.035 Total time: 0.035

[Makoto]

My server setup isn't all that special. :tongue:

My primary production servers uses this as its hardware configuration,

Intel Xeon E3-1270v2

16GB ECC DRR3 Memory

2x128GB SSD (Software RAID 1)

3x1TB Enterprise Grade 7200RPM HDD (Hardware RAID 5)

And my development server is pretty much the same, except it has a 4x2TB RAID 5 array.

My primary forum handles ~9,000-10,000 visitors with around 110,000-130,000 pageviews and 500-1TB of download traffic a day. (300-400 users online in a 15 minute window at peak hours)

So nothing too insane. Server load usually idles around 1.00, even at peak hours and with other services and processes running in the background. In general it handles my current traffic demand with ease, and I'm happy with it.

The server itself runs using the latest development release of Nginx (currently 1.5.8) compiled with SPDY and PageSpeed support. For the database backend, I run the latest stable release of MariaDB and use XtraDB for most of my tables and MEMORY for the session table. I also have the entire database loaded in the InnoDB buffer pool, so the database server alone consumes about ~4.5GB of memory.

I could list a page full of various tweaks I've made to my servers though. Most of them probably won't mean much on their own. Eg; setting vm.swappiness to 0 and ensuring TCP slow start after idle is disabled in sysctl.conf, mounting with data=writeback in fstab, using NSD over BIND (for both performance and security).. those are just a few completely random things that come to mind.

Like I said before, I just love learning and experimenting with things. That's where a lot of this comes from. :tongue:

I'm no professional though, so I'm sure my setup is far from perfect, but it serves me well.

[Aussie Cable]

And this is too a nearly blank html file that I put up just for the test, so no MySQL overhead to worry about.

Remote:

Connect: 0.304 TTFB: 0.359 Total time: 0.359

Local:

Connect: 0.171 TTFB: 0.198 Total time: 0.198

Wow those times are slow.

What are your server specs?

EDIT - My primary production server specs are:

2x Quad-Core Intel Xeon X5365 @ 3Ghz

32GB ECC RAM

6x Hot Swappable 146GB 15K SAS

2x Gbit NIC's connected to a 120Mbit cable modem

On a Dell PowerEdge 2950.

[Makoto]

And this is too a nearly blank html file that I put up just for the test, so no MySQL overhead to worry about.

Are you using Apache or Nginx?

[CheersnGears]

Apache

I just found a utility called YSlow that works with firebug and reported a whole host of page coding reasons for my page to load slow.

[Makoto]

TTFB doesn't include DOM, if it's taking 0.2 seconds just to return an HTML web page from your local machine, that means Apache in general is performing poorly for some reason.

Maybe your MaxClients value is too low and you're running out of free connections? It's probably something you should look into.

[GreenLinks]

Apache

I just found a utility called YSlow that works with firebug and reported a whole host of page coding reasons for my page to load slow.

Highly possible your apache configuration is not perfect.

[CheersnGears]

Is there anything like the mysqltuner.pl out there for Apache?

[CheersnGears]

9 idle workers and still getting the same times.

[CheersnGears]

By how under used my server is, it almost has to be an apache issue I guess.

top - 19:26:19 up 22:04, 1 user, load average: 0.07, 0.24, 0.25
Tasks: 153 total, 1 running, 150 sleeping, 0 stopped, 2 zombie
Cpu(s): 0.5%us, 0.1%sy, 0.0%ni, 99.3%id, 0.0%wa, 0.0%hi, 0.1%si, 0.1%st
Mem: 3819160k total, 3610272k used, 208888k free, 167912k buffers
Swap: 1473528k total, 80096k used, 1393432k free, 2135740k cached

[Grumpy]

Is there anything like the mysqltuner.pl out there for Apache?

No... but there are hoomans who respond here with higher quality advice than mysqltuner does...

[yacenty]

here is log from my production:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  453k    0  453k    0     0   358k      0 --:--:--  0:00:01 --:--:-- 1731k
Connect: 0.001 TTFB: 1.206 Total time: 1.265

what should we tweak?

for gallery page we have:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  138k    0  138k    0     0   306k      0 --:--:-- --:--:-- --:--:-- 8794k
Connect: 0.001 TTFB: 0.437 Total time: 0.451

[yacenty]

Kirito - do you use any php cache?, built in nginx like .nino. described or any other like varnish?

[Makoto]

Kirito - do you use any php cache?, built in nginx like .nino. described or any other like varnish?

Just APC. I've never touched Varnish.