Add Expires Headers when uploading to S3 to improve caching

[The Old Man]

I've been doing a lot of testing with S3/Cloudfront hosted files of late, trying to improve my GT Metrix Pagespeed scores.

Gzipped files in particular uploaded via IPS 4.3 to S3 are flagged by GT Metrix as lacking Specify a Vary: Accept-Encoding header - The following publicly cacheable, compressible resources should have a "Vary: Accept-Encoding" header. This results in a fairly large drop in score especially if both JS and CSS files are missing these headers.

Rather confusingly, you can't actually fix this by adding Vary: Accept-Encoding via the Cloudfront Settings as the advice suggests. I've tried every combination and method of S3/Cloudfront Settings, I can think of or find.

You CAN however fix this every time by manually adding simple Expires headers to those affected files over at S3, but you must invalidate your Cloudfront Distribution cache (and then finally at Cloudflare if using it).

 

Add the missing Expires header, changing the Cache-Control header to match if necessary/you prefer:-

You can see here I've changed caching and time before Cloudfront checks for an updated file to 8 days rather than 1 year, the minimum required by GTMetrix to pass this test on a cachable static JS file.

 

Remember that you must invalidate your Cloudfront Distribution cache after editing the Expires headers either individually or via the parent folder (and also at Cloudflare if using it).

I've tested this solution on all of my sites and it works, you can get an A97 or higher Pagespeed rating depending upon what else you do and external 3rd parties aren't affecting your score.

Adlago would be proud...

 

However whilst this manual intervention works okay for longer retention files like images etc that won't change, the issue is that when and if your IPS recaches itself (or you do it via the Support Tool), IPS uploads brand new cached JS/CSS files to S3. The next time you or GTMetrix bench-tests your site, the score (and performance and hence potential SEO ranking) falls. Then you have to manually add the Expires headers back to those affected temporary cache files.

We're now receiving A(100) for this recommendation!

 

Now Adlago would be even prouder, A97 rating admittedly not higher, but I'll take a 50% boost in page loading speed! 

The permanent solution, aside from getting stuck in this infinite loop (or writing a Node.js script at LambdaEdge to do it automatically when triggered on every request), is if IPS would automatically add the Expires Headers to resources each time it uploads the files for us to S3, via the S3 Restful API. Secondly, have the option to add headers or amend the values would be the icing on the cake.

[Matt]

Um. I'm going to tag in @Charles

[The Old Man]

 

[asigno]

This would be great to implement, I have the same issue with Cloudfront but don't have the time to keep manually changing things.

[The Old Man]

Yes, it's a truly royal PITA. I'm so fed up of manually adding these missing headers and invalidating Cloudfront and Cloudflare caches every time the theme JS or CSS is uploaded ,and multiply that by the number of sites you have.

Have recently tried adding exposeheaders tags to the CORS setup on my S3 bucket, but it made no difference. Unless the Expires tags are manually added to the files Cloudfront or S3 won't forward them to the visitors browser. Adding them automatically when they being uploaded is still the best potential option.

[Matt]

@Charles had some thoughts on this.

[The Old Man]

Sounds positive, thank you. 🙂

[Joel R]

On 1/21/2019 at 4:39 AM, The Old Man said:

Sounds positive, thank you. 🙂

You don't know how @Charles thinks though LOL.  

[thetrials]

Still curious about this. Just moved everything over the S3, Cache-Control headers are there but not expires...

[The Old Man]

Yes, they are not showing here on 4.4 either so I don't think Charles has waved his wand yet. In fact the GTM score here is in the low 80's.

https://gtmetrix.com/reports/invisioncommunity.com/Fp7jIA6X

The 4.4 changelog says that improvements to caching have been made, especially for guests, that must be something else.

[catbreadbat]

Can we get an update on this? Having just moved some static assets over to S3, I am facing the same concern from site speed testing tools.

[The Old Man]

Having spent some time and effort on a reliable working solution for this, I intend to release a configurable Marketplace plug-in soon for it. It will be my first plug-in, so please bear with me. Will update this topic once done!

[sudo]

This is one of the reasons I use Cloudflare over Cloudfront infront of things as a CDN, so so much easier to configure and force headers.

[The Old Man]

I actually use both Cloudflare and Cloudfront with S3.

[asigno]

@The Old Man did you get any updates from @Charles on this?

[The Old Man]

Hi, no but I wasn't actually expecting to. However I've now been able to create a plug-in which does the job.

I've now tested it on all of my 4 of my IPS Community sites using S3 in the last 48hrs and it works fine.

I'll aiming to upload it to the Marketplace later today, and will post here if and when it's approved.

 

[asigno]

@The Old Man Awesome look forward to it, thanks for working on building this. I'm a little puzzled though why this isn't standard with the AWS integration.

[The Old Man]

Thanks @asigno. I don't know but I've uploaded it to the Marketplace earlier this evening, so hopefully once approved it will save some time and repetition manually adding them for those who want the extra caching headers. Sorry for the delay, I wanted to do some additional testing.

 

 

[The Old Man]

Now available at an IPS Marketplace near you...