CloudFlare settings

[Randy Calvert]

Just now, marklcfc said:

My topics need to be up to date with latest posts available to guests

In this scenario, you're better off honestly without caching.  🙂 

Think of it this way...  even a site that shows 300 visitors online is 300 users over the last 30 minutes.  That's an average of 10 users a minute on AVERAGE across your entire site.  They're all not going to be sitting on that one page to be cached...  they're viewing different topics, or sub forums, or looking at your other various pages.  You're spending more server efforts putting a page into cache for that subsequent page view when 95% of the time, no one is ever going to get that same page within the next 29 seconds before it expires.  

[marklcfc]

I've checked other sites using these cache rules and they seem up to date and viewing as a guest does not appear hours behind in terms of topics/posts so not sure if I'm missing something. I've only just changed nameservers so I guess I'll soon find out.

Edited October 28, 2022 by marklcfc

[Randy Calvert]

11 minutes ago, marklcfc said:

I've checked other sites using these cache rules and they seem up to date and viewing as a guest does not appear hours behind in terms of topics/posts so not sure if I'm missing something. I've only just changed nameservers so I guess I'll soon find out.

Remember...  there are 2 options...  override and respect.  You can set a manual cache time, but you have to do it in 1 hour increments.  If you're respecting origin, IPB's cache control headers are set to 900 seconds.  Meaning would be delayed NO MORE than 15 minutes.  Here's the exact header values being passed:

As you can see, the max-age value is 900 seconds.  This means the LONGEST it would be delayed is 15 minutes...  assuming someone posted something new just seconds after the original topic page was cached.  Again... that's a worst case scenario.  The more common case is a few minutes.  

Edited October 28, 2022 by Randy Calvert

[marklcfc]

What should Browser Cache TTL be in Caching > overview

It defaults to 4 hours but there's an option for Respect Existing Headers

Edited October 28, 2022 by marklcfc

[Randy Calvert]

Pick "Respect Existing Headers".  

[marklcfc]

It's been a few hours since I changed nameservers, just looking on SSL/TLS it seems there is still some not secure connections, what would cause that? I haven't changed any of those https settings inside cloudflare yet as was suggested not to.

 

[marklcfc]

Also does this look right, notice a fair few cache misses

[Randy Calvert]

It’s normal to have some small portion of traffic not be SSL. The very first request might be via HTTP that gets automatically kicked over to HTTPS. No worries there. 

You’ll never have a 100 percent cache hit rate. Some content is uncachable (base pages for logged in users for example) or the 15 min TTL is expired for guests.  But images, style sheets and some JS are catchable for everyone. 

A single page load might have 100 objects on it. Like 90 of those objects might come from cache. 

Remember… this offloads most of your static content as well so your server does have to serve that and gives users better performance. 

Edited October 28, 2022 by Randy Calvert

[marklcfc]

Anything worth turning on in Speed > Optimization 

[Randy Calvert]

I personally was able to use the "Auto Minify" JS, CSS, and HTML without issue.  This is somewhat dependent on your theme though if it will work without problems, so if you attempt to use it:

• Enable feature
• Clear cache
• Wait a minute
• Browser around your site a minute
• Repeat prior step to make sure nothing looks unusual

In addition to Auto-Minify, I personally also enabled Brotli and Early Hints.  Brotli is a compression algorithm that is designed to make objects smaller before they're transferred from the server over the internet.  Early Hints is not going to be a big game changer in its current state. But the future state of the dynamic early hints will make more of a difference.  It does not hurt in any way to have it enabled.  ANY little benefit that does not have a downside is good.  🙂   

[Charles]

On 10/28/2022 at 9:30 AM, marklcfc said:

I set it to 30 seconds else guests ended up not seeing the latest posts for 5 minutes. 

Does this mean guests will not see the latest posts on cloudflare now? if so it’s not an option  

Honestly, if you only ever had it set to 30 seconds your site will actually be faster if you did nothing. Our old guest page caching added overhead to your server. With a cache timeout of just 30 seconds your server was working harder than it would than if you just served pages directly.

I would still recommend a CDN but you should be fine either way and your MySQL will be on vacation.

[marklcfc]

1 hour ago, Charles said:

Honestly, if you only ever had it set to 30 seconds your site will actually be faster if you did nothing. Our old guest page caching added overhead to your server. With a cache timeout of just 30 seconds your server was working harder than it would than if you just served pages directly.

 

I've had it at 30 seconds for the past 5 years, I suppose I didn't completely understood how it worked. My impression was that guests cached page would update every 30 seconds, if I had it for 5 minutes then it would only update every 5 minutes and they wouldn't see the latest posts.

[Randy Calvert]

57 minutes ago, marklcfc said:

I've had it at 30 seconds for the past 5 years, I suppose I didn't completely understood how it worked. My impression was that guests cached page would update every 30 seconds, if I had it for 5 minutes then it would only update every 5 minutes and they wouldn't see the latest posts.

That’s what it does… but in most cases, guests don’t need a true “live” view. 

By setting it to 30 seconds, if you did not have tons of users in the site, that cached page would never get re-used. You spent a bunch of server resources writing that page to disk that never got used because no one was really accessing it within the next 30 seconds. 

Think of it this way… Google bot comes to your site. It starts on the homepage. That’s now told your server to not just deliver that page to Google but make a version of it for offline use and to hold onto it for 30 seconds. The rest of the time, Googlebot was not on your homepage. They were on the inside content of your pages. Triggering the same thing all over again creating a cached version that no one else ever accessed in the next 30 seconds. 
 

Is there some value gained by a guests like Google seeing data “real time”?  Having content delayed a few minutes won’t make a huge difference for them. 

However if you honestly did feel it was critical… Instead of trying to cache at the server level that low, the page experience would be faster by not caching that base page at all. 

With a CDN, you also have the fact that each CDN POP is unique. Cloudflare has over 300 locations. This means if you have someone accessing content in LA, its cache timer is separate from the servers in Miami. (Each has its own timer based on how long it’s been since that specific server last accessed it.) 

Super short cache periods for base pages only help if you’re operating a hugely busy site like CNN, Wikipedia, etc.  

Also remember a CDN has an extra benefit over server caching… the CDN can cache most all objects.  This means images, style sheets, JS, etc all can be cached for DAYS whereas the base HTML page might be cached for a few minutes.  This can reduce origin bandwidth by 50-90 percent depending on how dynamic the base pages are. 

Edited November 1, 2022 by Randy Calvert

[Charles]

11 hours ago, marklcfc said:

I've had it at 30 seconds for the past 5 years, I suppose I didn't completely understood how it worked. My impression was that guests cached page would update every 30 seconds, if I had it for 5 minutes then it would only update every 5 minutes and they wouldn't see the latest posts.

Yes, you have it right but with only a 30 second cache time you are getting no benefit. You would do better to have no caching.

The old guest page caching would:

1. Check MySQL DB if cache exists
2. See it is expired, delete from MySQL
3. Render a page as normal
4. Insert row in MySQL with rendered output
5. Serve to browser

You could have skipped all but step 3. A CDN will of course skip all steps because it never hits your actual server.

[marklcfc]

Fair enough, hopefully it works out better without it then as I've set up Cloudflare but once again no real idea how I am supposed to set it up but have gone by what's been posted in here

[marklcfc]

Pretty sure my site is slower since changing to Cloudflare.

[Randy Calvert]

8 minutes ago, marklcfc said:

Pretty sure my site is slower since changing to Cloudflare.

You can test it pretty easily by changing Cloudflare's status to OFF.  In the DNS settings, change the orange cloud to grey.  

However instead of just saying "pretty sure", I would recommend using some sort of performance monitoring solution for a period of time with Cloudflare on and off to get actual supporting data.  Guessing based on "feel" is a bad judge of efficacy.

Edited November 8, 2022 by Randy Calvert

[marklcfc]

9 minutes ago, Randy Calvert said:

You can test it pretty easily by changing Cloudflare's status to OFF.  In the DNS settings, change the orange cloud to grey.  

 

It just feels sluggish. Where abouts is this setting? In DNS I only see the orange cloud for the proxied A DNS entries. 

[marklcfc]

I don't know, something weird is going on. I unplugged my ethernet cable and back in and my site is fast again, this happened yesterday I did the same thing and everything sped up.

It remained slow on my phone, so I turned wifi off and then back on on iPhone settings, now it has has sped up on my phone. I'm sure this time tomorrow it will be sluggish again, pretty sure this only started to happen since I changed to Cloudflare. If its happening to me on multiple devices it must be the same for everyone else

Edited November 8, 2022 by marklcfc

[Randy Calvert]

1 hour ago, marklcfc said:

It just feels sluggish. Where abouts is this setting? In DNS I only see the orange cloud for the proxied A DNS entries. 

The orange cloud…. Click it to turn it off. The proxied status is what routes requests through CF servers. If it’s off, it sends requests straight to your server instead without doing anything to the request. 

32 minutes ago, marklcfc said:

I don't know, something weird is going on. I unplugged my ethernet cable and back in and my site is fast again, this happened yesterday I did the same thing and everything sped up.

It remained slow on my phone, so I turned wifi off and then back on on iPhone settings, now it has has sped up on my phone. I'm sure this time tomorrow it will be sluggish again, pretty sure this only started to happen since I changed to Cloudflare. If its happening to me on multiple devices it must be the same for everyone else

That actually indicates it’s NOT a Cloudflare problem. Both instances were situations where the local home network was restarted and fixed the issue. 

Unplugging your Ethernet cable disconnected it from the local router and restarting wifi the same thing… so this sounds like an issue within your home network not CF. Again this is where external monitoring would more accurately prove what is going on. 

[marklcfc]

3 minutes ago, Randy Calvert said:

The orange cloud…. Click it to turn it off. The proxied status is what routes requests through CF servers. If it’s off, it sends requests straight to your server instead without doing anything to the request. 

Are you talking about these? I have to edit them and turn proxied to off?

 

[The Old Man]

You don't want to do that. You'll essentially be losing all the performance and security benefits of Cloudflare and also giving away your origin's IP address, akin to pausing the site on Cloudflare or disabling the site.

The problem is there is no specific Cloudflare API support in IPS, no documentation guidance on correct replacement of local guest page caching, and we're being told the software will set the correct headers. It's never done that properly unless this has changed. Even then, Cloudflare may choose not to honour it and cache it anyway (under some circumstances), which is why utilising CF Workers to manage caching gives greater peace of mind,

Is there any reassurance and testing been done to date so far that CF won't cache Moderator CP pages, PM's, Commerce baskets, account details etc.

The problem has always been that you can't just set CF to cache everything (so that it does HTML which it normally doesn't, unless you set it via a Page Rule) combined with 'Bypass on cookie' enabled, because that feature was always been available only for Pro accounts upwards.

I'm going to experiment with the suggestions so far myself, Opentype mentioned some positive results. CF seems to be separating it's caching rules and increasing them which is great as the free accounts were always limited to just 3.

Edited November 9, 2022 by The Old Man
Expanded

[Randy Calvert]

He's saying that he suspects CF is slowing his pages down instead of helping him.  So if that's the case, the only way to confirm that assumption is to test it.  Testing needs to be done with it both on and off.

Regarding exposing the origin... it was exposed before being moved to CF.  It's not adding any additional risk that was not already there a few weeks ago.  

6 minutes ago, The Old Man said:

Is there any reassurance and testing been done so far that CF won't cache Moderator CP pages, PM's, Commerce baskets, account details etc. The problem has always been that you can't set CF to cache everything with bypass on cookie enabled, because that feature was always Pro account upwards.

Regarding testing caching...  PMs, ModCP, etc...  ALL of those areas require being logged in.  If you're logged in...  base pages are not being delivered from cache if you've set the cookie values provided by IPS to be ignored.  And you can do it with a free plan.  It's been explained how to do it already. 🙂 

7 minutes ago, The Old Man said:

The problem is there is no specific Cloudflare API support in IPS

CF API support is not needed.  Create the rules defined by IPS already for what not to cache (some specific path values along with 2 cookie values) and you don't have to do anything else.  

8 minutes ago, The Old Man said:

no documentation guidance on correct replacement of local guest page caching,

99.99999% of people don't need local guest page caching.  Especially those folks who are setting cache to less than 10 minutes.  Unless their site has thousands of concurrent users, it's not needed honestly.  So in most cases, the solution is just to do nothing and let the old feature be deprecated.  For those that want the value of caching, there are CDNs.  And yes, instructions have been provided.  🙂

9 minutes ago, The Old Man said:

and we're being told the software will set the correct headers. It's never done that properly unless this has changed.

The cache headers don't specify what can be cached or not.  Those are defined in the cache rules.  The headers specify how long objects can be stored in cache.  For example, base pages should be cached for 15 minutes while CSS/JS should be cached for a year (since IPS automatically renames these files each time a theme is generated), etc.

IPS has been using proper cache control headers for awhile now.  In fact, it's been in use on this community here for several years now.   

Here's a sample of the headers from an image on this site:

accept-ranges:bytes
age:175880
alt-svc:h3=":443"; ma=86400
cache-control:public, max-age=31536000
content-encoding:gzip
content-length:58464
content-type:text/css
date:Fri, 04 Nov 2022 14:12:53 GMT
etag:"7826cec1f91163ffee9f5134a50a4658"
last-modified:Thu, 03 Nov 2022 22:25:00 GMT
server:AmazonS3
via:1.1 aeb4230d4287e12c8862574307ac71a2.cloudfront.net (CloudFront)
x-amz-cf-id:amIQ-EX7iZSTMXVsK88VuD-MAcUupJw7TgKZENPBS4wENhxh17anvQ==
x-amz-cf-pop:MIA3-C2
x-cache:Hit from cloudfront

The cache-control line specifies the type of content and how long it can be cached.  The content encoding line says to try to gzip the output to reduce the transfer size.  The etag line allows the CDN to better match if content has been changed by matching an ID value of the file instead of trying to get a file and compare it after the fact.  

[G17 Media]

IPS sets the headers just fine!

Provided your guests don't change the theme. And provided they don't use a different language. 🤔

Edited November 9, 2022 by G17 Media

[The Old Man]

2 minutes ago, Randy Calvert said:

He's saying that he suspects CF is slowing his pages down instead of helping him.  So if that's the case, the only way to confirm that assumption is to test it.  Testing needs to be done with it both on and off.

I'm not trying to antagonise you, I recognise you spend a lot of time helping and advising around the forums, but you've suggested a sledge hammer to crack a walnut, on someone else's server who at the end of the day you/we don't know anything about.

I'm suggesting that there are probably more cautious or better ways to go about this by enabling or deactivating specific features and functionality. You have to remember that other people are reading this as well. 

With IPS being a paid for solution (and not cheap!) which is officially supported, there should be an official guidance page in the help section, preferably before they started to deactivate features! We shouldn't be stressing over this now, just because IPS want people to migrate to their preferred eco system and they have to some extent left us twisting in the wind. Our CDN does caching doesn't mean a lot to those not using Cloudfront.

20 minutes ago, Randy Calvert said:

Regarding testing caching...  PMs, ModCP, etc...  ALL of those areas require being logged in.  If you're logged in...  base pages are not being delivered from cache if you've set the cookie values provided by IPS to be ignored.  And you can do it with a free plan.  It's been explained how to do it already.

In theory perhaps. IPS has never always set the correct headers and cookie values for all areas of the site, look at the pigs ear that was made of S3 storage. I seem to recall that cookies were included with a response, the Cloudflare CDN wouldn't cache a response. That's why you needed the bypass on cookie option, which was fine if you were not on the free plan.

You're including headers from Cloudfront and comparing the end result with Cloudflare, which isn't the same type of CDN at all.