CloudFlare settings

[marklcfc]

So just clear that line but continue to use Redis?

[The Old Man]

Yes, see if that makes any improvement for you.

[The Old Man]

You may have to create a new constants file via the AdminCP if it doesn't take. (Via the save in the Data Storage tab).

I've just checked and found it again in one of my sites that was working but caching it for 3600 seconds, so that was indeed the reason why the other site wasn't doing it at all, as it was set to zero, this one was at 3600:

Quote

 

<?php

\define( 'REDIS_ENABLED', true );
\define( 'STORE_METHOD', 'Redis' );
\define( 'STORE_CONFIG', '[]' );
\define( 'CACHE_METHOD', 'Redis' );
\define( 'REDIS_CONFIG', '{"server":"127.0.0.1","port":6383,"password":""}' );
\define( 'CACHE_PAGE_TIMEOUT', 3600 );
\define( 'SUITE_UNIQUE_KEY', '0730e84dbb' );

 

It's definitely overriding the time setting for Cache Control and Expires that we were expecting to see, having set CFlare to respect the origin's settings.

Edited November 16, 2022 by The Old Man

[marklcfc]

Great work @The Old Man I never thought I was going to solve that one 😄!

Although I feel now I've seen how it should be, my fears are true in that its just not updating for the guest.

See example, live site:

Cached guest site

General chat, it is not updating. As I post this now, its been 5 minutes since the general chat last post was made, but its not changing for the guest. This is why I previously set a guest page cache of 30 seconds.

So now I have to turn it off 😞

 

Edited November 16, 2022 by marklcfc

[Randy Calvert]

 

That means the content could be up to 900 seconds (15 minutes) delayed.  If that does not work for you, don't cache the base pages.  Let CloudFlare only serve the associated images/stylesheets/javascript from cache.  

A value of 30 seconds literally has ZERO value.  In fact, it will hurt you more than it helps in a majority of cases unless your site is CNN, Facebook, or some hugely trafficked site.  

Edited November 16, 2022 by Randy Calvert

[marklcfc]

4 minutes ago, Randy Calvert said:

That means the content could be up to 900 seconds (15 minutes) delayed.  If that does not work for you, don't cache the base pages.  Let CloudFlare only serve the associated images/stylesheets/javascript from cache.  

 

I think that's what I want now. I've deleted the cache rules, will this achieve that?

Edited November 16, 2022 by marklcfc

[Randy Calvert]

1 minute ago, marklcfc said:

I think that's what I want now. I've deleted the cache rules, will this achieve that?

Delete the IPB caching rule and just leave the rest alone.  Cloudflare automatically delivers the rest of the objects from cache when possible.  It's always been hitting from cache for other non-base page objects since you first onboarded.  For example:

[The Old Man]

1 hour ago, marklcfc said:

Great work @The Old Man I never thought I was going to solve that one 😄!

Although I feel now I've seen how it should be, my fears are true in that its just not updating for the guest.

I'm glad we found it, it's been driving me nuts too. I had to clear the CF cache after removing that rule from the second of four sites that had it and it's working as intended. These extra caching rules that CF now provide free accounts are the dealbreaker here.

So looking at your cache time now, 900s/15 mins, and it's not revalidating each time it loads for a guest to check if the page has changed, but it is doing what IPS have set it to. For now though, what you are seeing is what @Day_ posted, BYPASS replaced by DYNAMIC, so perhaps it's not fully configured if it was enabled when I took this screen shot:

As previously posted by another chap in this thread...

Quote

Be aware that when you select Eligible for cache in Cache Rules, this is equivalent to cache everything cache level. If you do not enable eligible for cache, the expected behavior is the standard cache level present in Page Rules.

As I understand that, and this quasi-outdated from the documentation:

Quote

DYNAMIC

Cloudflare does not consider the asset eligible to cache and your Cloudflare settings do not explicitly instruct Cloudflare to cache the asset. Instead, the asset was requested from the origin web server. Use Page Rules to implement custom caching options.

assuming you still have Jim's Cache Rule enabled and still set to Eligible for Cache, you need to make sure Respect Origin is enabled further down or you have selected an alternative caching options, as it is falling back to the old Page Rules for what to do; and I think because your Page Rule is probably empty and you don't have one with the Cache Everything set, it's using the default action not to cache html pages, just static assets like CSS, JS, images, fonts, etc only if they are stored on your server and not remotely like on a Google CDN.

I'd check the Caching Rule is set correctly but I think you could try the other options to get CF to revalidate. If you were setting it manually, adding the oddly mis-titled 'No-Cache' option would do that.

Edited November 16, 2022 by The Old Man

[marklcfc]

I’ve removed those cache rules as of my previous post so the screenshot would have been after I think. I think having no cache rules for now will give me less of a headache, assuming performance is ok during busier times. 

I’ve currently got no cache rules or page rules set up

8 minutes ago, The Old Man said:

just static assets like CSS, JS, images, fonts, etc

I’ll settle for this I think, which is what it’s been doing the whole time whilst set up incorrectly I believe

[The Old Man]

Ah, thanks for the confirmation, yes having no rules at all will cache the usual assets only.

Edited November 17, 2022 by The Old Man

[marklcfc]

Now I'm wondering whether I need the Pro version I signed up for or just go free as I'm doing nothing but that 😄

Edited November 17, 2022 by marklcfc

[Randy Calvert]

You did not need the Pro version at all.  Everything talked about can be done on the free plan.  

https://www.cloudflare.com/plans/#overview

[The Old Man]

Hi @marklcfc

I'm still researching this along with getting Cloudflare to cache non-default theme pages for guests seperately based on the ips theme cookie (the Cache Key settings are missing the documented Cookie field preventing this).

Thanks to the new Caching Rules , it looks like you can overide the default via the CF settings, but ONLY to increase the time beyond what IPS sets by default (the 900 secs/15mins) for 2hrs up to a year. You used to be able to and probably still can select a shorter period in the default CF settings.

I think as you have mentioned you want to cache it for less 900s so guests always see the latest topics, but I think you'd be better off getting it to revalidate before still serving the 900 sec cached page. That way the guest should always see any new content within a few seconds.

To do that, you need to change the IPS cookie header, do it via your server, or via Cloudflare. 

I personally can only see a need for the Pro plan to get access to on the fly image resizing and conversion, but its based on your needs. I sell about 2-3 themes a year, so I can't afford it!

Edited November 17, 2022 by The Old Man

[marklcfc]

The only reason I went Pro was for some support if needed setting it up, and it says image optimization and accelerated mobile pages (a large % of my visits are mobile)

[Day_]

@marklcfc does this mean I'm off the hook now? 😬

[RevengeFNF]

Anyone here tried to use Cloudflare R2 that is S3 Api compatible and a LOT cheaper?

[Randy Calvert]

How much storage are you actually using?  It technically works, but the APIs had been slower in my personal experience. I would see gallery uploads randomly timeout, etc.  Unless you have tons of external storage, I would most likely just use S3 and put CF in front of it as a CDN so that you dramatically reduce your AWS bandwidth.  I would also make use of the S3 intelligent tiering to reduce cost of storage that is rarely accessed.  

[The Old Man]

No, not used it at all 🙂

[Chris027]

On 11/19/2022 at 3:05 PM, RevengeFNF said:

Anyone here tried to use Cloudflare R2 that is S3 Api compatible and a LOT cheaper?

Was just looking into R2 tonight. Would love to use it over Amazon S3. 

[SeNioR-]

@marklcfc Well done with the cache configuration 👍Your site loads much faster now, but I noticed quite a lag with the "Who's Online" widget and the online status in the topic which is a bit pointless now as it takes 15 minutes to update.

I wonder if (with your present configuration) you could now set "Cache sidebar, header and footer blocks" to 0 because the whole page (with the all widgets) is cached by CF now.

Plus, I think you can add some new CF rules like /discover/, /search/ and sitemap.php to avoid situations like in the picture below where there is a problem with loading reactions icons in search results.

[marklcfc]

@SeNioR- I don't have any cache rules set now, as mentioned a few posts up, since it wasn't updating for a long time for guests. So now I am using standard Cloudflare cache with not alot else changed or added.

So I assume the suggestion of setting the sidebar/header etc as 0 was only if I had those cache rules set up? 

Concerned about those images missing though, any idea what would be causing that?

[SeNioR-]

1 hour ago, marklcfc said:

I don't have any cache rules set now, as mentioned a few posts up

Yes, but you still have the cache set to 900 seconds (15 minutes) in the headers for guests 😉 

1 hour ago, marklcfc said:

So I assume the suggestion of setting the sidebar/header etc as 0 was only if I had those cache rules set up? 

Yep.

1 hour ago, marklcfc said:

Concerned about those images missing though, any idea what would be causing that?

It's possible that it was a one-time situation, because the icons display correctly now.

Edited December 3, 2022 by SeNioR-

[The Old Man]

Hi,

I noticed it's possible to set a smaller time in Cloudflare using the Transform Response Header options, no need for any IPS modification it can be done on the fly to overwrite the default 900 values. By using the same regular expression (Jim's) as before to configure the Transform Response Header, then set the TTL to whatever you want for the Browser max-age and Edge s-maxage values, like 60 seconds or whatever you wanted.

...

So after a few weeks of trying, I still can't find a reliable method to get the guest page cache to work with themes like it used to.

Yes, on the free Cloudflare accounts thanks to the new improved Cache & Transform Rules and IPS now setting different headers, we can use their CDN to provide a guest page cache.  

Another positive is it requires almost no resources/there's no overhead server-side no matter how short you might cache the pages for, because it's taken over by the CDN and it's not a traditional push CDN.

Guests however can't view more than the 1 default theme anymore unless they go to uncached page a page (such as a product page within the Commerce store), then they will see their selected theme but it's gone again when they go back to say the forums. Now I know that it's actually the same with Wordpress, only 1 theme in use at a time, but forums and apps within Invision Community are more, and have supported different themes for guests as well as members since day 1. It's a nice to have, and I want to keep it!

I think it would actually be be possible right now with the new Cache & Transform Rules, but when you start looking in more depth at the docs, CF have gone and move the goal posts on us again by not allowing free accounts access to the necessary functionality:-
 

Quote

Availability notes

• Access to ip.geoip.is_in_european_union, ip.geoip.subdivision_1_iso_code, and ip.geoip.subdivision_2_iso_code fields requires a Cloudflare Business or Enterprise plan.
• Access to http.request.cookies field requires a Cloudflare Pro, Business, or Enterprise plan.

AFAIK for the 2 people people including me wanting guest theme caching (I jest), I think we need to set the theme cookie with a value in the response to match the current value of the theme cookie (say '43') sent in the client's request header. I think something like setting a cookie in the HTTP Response Header Modification section to return a matching value

Quote

any(http.request.cookies["IPS4_theme"][0] 

would do it.

For that, unless CFlare give us access to the set the http.request.cookies dynamically, we would still need to write Worker code, use the API or use the URI path to include the theme id, not nice, or Vary on the IPS4_theme Cookie.

I'd like a working solution to this caveat. It's a shame to have to pay extra for Pro just for the ability for guests to use more than the current default theme or a dark mode. I know IPS view list as an acceptable loss, but for me it's not ideal.

[Randy Calvert]

The stance I took personally was if you want a personalized experience such as being able to pick a theme, you should register an account. Otherwise you get the default theme only. 

[marklcfc]

Had my first proper busy spike last night and the site crashed, got a load of internal server errors and had lots of these in the logs

2023/01/10 21:20:49 [alert] 127868#127868: *28491671 socket() failed (24: Too many open files) while connecting to upstream

Pretty sure if I had guest page caching still it wouldn't have been as big a problem as it was, first time in 4 years the current server has ever been properly overloaded which I could see coming as soon as Guest page caching was removed. The amount of members and guests connecting to the site at the same time must have been pushing 2,000.

Not sure what I can do now as Cloudflare hasn't helped prevent this (its just set up with default settings, as mentioned earlier in the topic I tried to replicate the guest page cache but it never updated the site for guests for a long time) 

Edited January 11, 2023 by marklcfc