Cache page output for guests and core_output_cache table

[ABGenc]

I have been experiencing server load issues and users were complaining about it and was not able to find the cause till I got a disk warning today. I realised that a database called core_output_cache had reached 18GB of size ~80.000 rows ( where my all remaining tabled were only 2 GB in total )

Since the size was so big that I could not take a look into it but was aware the server and mysql was wasting too much process. Risking and thinking it contained cache data I dropped the table and support tool fixed it but still the records were coming very fast ( size had grown 30 MB in a minute and almost 100 rows were written )

Looking into it I saw some page cached were written but the expire dates had previous times. Thinking this could be the issue I disabled Cache page output for guests option and now the server is much better on server load and mysql performance.

Any known issue about this option and table ?

[b416]

There is a discussion about that here : 

 

For me, the solution was to use redis as Data Storage Method and to add this undocumented option to constants.php  :

\define( 'OUTPUT_CACHE_METHOD', 'Redis' );

My Cache page output for guests is now set to 5 minutes and everything works just fine.

[ABGenc]

Thanks for the info @b416I had the Redis option enabled for the guests but had not added the config you have mentioned to constants.php you have mentioned. Will try that asap 😉

[ABGenc]

@b416 thanks again, I have followed the suggestions and had a remarkable decrease on response time and CPU / memory utilization.

[Adlago]

This page is working poorly - this is my personal opinion. There are many unclear questions.
If the first guest is out of time cache, there is a slow net connectivity, or an outdated browser, or slow mobile connectivity - what data is stored on this guest cache page? And what will all subsequent guests load up to when this new cash is expired?
In my opinion, this page should not be used.

[sudo]

1 hour ago, Adlago said:

This page is working poorly - this is my personal opinion. There are many unclear questions.
If the first guest is out of time cache, there is a slow net connectivity, or an outdated browser, or slow mobile connectivity - what data is stored on this guest cache page? And what will all subsequent guests load up to when this new cash is expired?
In my opinion, this page should not be used.

Quite confused by this, the cache mentioned here is server side, it doesn't have anything to do with client side/browser cache. The server caches the page and subsequent guest views use the generated cached page.

[Adlago]

15 minutes ago, sudo said:

Quite confused by this, the cache mentioned here is server side, it doesn't have anything to do with client side/browser cache. The server caches the page and subsequent guest views use the generated cached page.

Why do you view the server as an ideal item? Yes, the server caches this page, but this page is not generated at the request of the server, but at the request of a guest.

[sudo]

4 minutes ago, Adlago said:

Why do you view the server as an ideal item? Yes, the server caches this page, but this page is not generated at the request of the server, but at the request of a guest.

It reduces server load and subsequent views will be send to the end user/other guests quicker. Page headers that set cache time expiring on the browser are a separate thing and I don't know if they are set client side for guests by ipb or not.

I do agree about browser side caching being a useful tool but thats a bit off topic when this is talking about server side to reduce load. Server side caching is always a good thing as even if the guests browser hasnt cached or hit the page before they get a speed benefit regardless.

[Adlago]

24 minutes ago, sudo said:

It reduces server load and subsequent views will be send to the end user/other guests quicker. Page headers that set cache time expiring on the browser are a separate thing and I don't know if they are set client side for guests by ipb or not.

I do agree about browser side caching being a useful tool but thats a bit off topic when this is talking about server side to reduce load. Server side caching is always a good thing as even if the guests browser hasnt cached or hit the page before they get a speed benefit regardless.

And what happens to multilingual sites?
What does an English user receive if a guest has used a local language before and this is cache page?
What happens when a user with an already downloaded guest page cache reloads a page after a timeout cache and a new cache page is in another language?

[Adlago]

And one more thing - if you do "Something isn't working correctly" - won't all guests have CSS and JS until the cache time expires?

[sudo]

27 minutes ago, Adlago said:

And what happens to multilingual sites?
What does an English user receive if a guest has used a local language before and this is cache page?
What happens when a user with an already downloaded guest page cache reloads a page after a timeout cache and a new cache page is in another language?

Woaw I have not digged into the source to see how it handles auto language selection normally let alone with a guest cache server side although it's certainly possible it could grab the cache for each bit of content depending on what the browser requests saving it having to load it from the database directly. I just dont know if it does that but the whole point of this is to reduce database calls and load by pulling precompiled pages from a faster cache.

12 minutes ago, Adlago said:

And one more thing - if you do "Something isn't working correctly" - won't all guests have CSS and JS until the cache time expires?

I wouldnt expect errors to trigger a cache as its not a load intensive thing. I dont know how that has to do with CSS/JS as thats client side. If the page loads properly they will get the correct CSS/JS for the page, if its an error it will load the correct CSS/JS for the error page.

[Adlago]

"Something isn't working correctly" changes csrf key in CSS and JS links. If a site uses apache for the primary server and nginx for the cache server, after "Something isn't working correctly" all guests receive a cache page from the nginx server, but with the old csrf key - ie. CSS and JS not functioning ...

[sudo]

15 minutes ago, Adlago said:

"Something isn't working correctly" changes csrf key in CSS and JS links. If a site uses apache for the primary server and nginx for the cache server, after "Something isn't working correctly" all guests receive a cache page from the nginx server, but with the old csrf key - ie. CSS and JS not functioning ...

This has nothing to do with the server engine being ran, this is php side caching control.

The CSRF key seems to be set in the page html not the css/js when I tested this page in incognito and the css/js paths are the same. That bit of HTML though that output the csrf key may not be cached and generated live but the bulk of the content of the page is which provides a server php gain in generation and mysql load. PHP can decide on which parts of the whole page are generated each time or which parts are pulled if cached, the same way how page blocks are cached but that doesnt mean the whole pages html is cached.

It may be better rather than theorising about problems if you actually tried some of this stuff out and found a problem. I am theorising a lot of what I am saying as I am not going to trawl through the source to confirm all the things you think dont/wont work.

[ABGenc]

By the way does anybody have an idea why the Cache (displayed in ACP Support ) is mostly 0 ?

 

[Martin A.]

23 hours ago, Adlago said:

And what happens to multilingual sites?
What does an English user receive if a guest has used a local language before and this is cache page?
What happens when a user with an already downloaded guest page cache reloads a page after a timeout cache and a new cache page is in another language?

It creates one version for each language, and another one for each theme.

22 hours ago, Adlago said:

"Something isn't working correctly" changes csrf key in CSS and JS links. If a site uses apache for the primary server and nginx for the cache server, after "Something isn't working correctly" all guests receive a cache page from the nginx server, but with the old csrf key - ie. CSS and JS not functioning ...

CSRF and cachebust keys are not the same. CSRF is used in forms, and other things that causes an action to something. This key is injected to the cache for each request, as this is unique for each visitor.

The cachebust in the cached output is not changed, nor is the cache dropped when running the support tool. @teraßyte posted a bug about this yesterday. 

And this is not about running any additional cache layers in your web stack, such as nginx or Varnish. All this feature does is taking the HTML generated from a guest visit, and storing it in either a database table, or in Redis, for any subsequent requests that may occur within the defined expire time (this defaults to 30 seconds). Any visits within this period will be served the same HTML as generated by a previous guest, saving a lot of unnecessary processing time. 

9 hours ago, ABGenc said:

By the way does anybody have an idea why the Cache (displayed in ACP Support ) is mostly 0 ?

Do you have this in your constants.php?

\define( 'REDIS_ENABLED', true );
\define( 'STORE_METHOD', 'Redis' );
\define( 'STORE_CONFIG', '[]' );
\define( 'CACHE_METHOD', 'Redis' ); /* Specifically this */

 

[ABGenc]

2 hours ago, Martin A. said:

Do you have this in your constants.php?

Yes I do. Here is my whole constants.php

\define( 'THUMBNAIL_SIZE', '130x130' );
\define( 'REDIS_ENABLED', true );
\define( 'STORE_METHOD', 'Redis' );
\define( 'STORE_CONFIG', '[]' );
\define( 'CACHE_METHOD', 'Redis' );
\define( 'REDIS_CONFIG', '{"server":"127.0.0.1","port":6379,"password":""}' );
\define( 'CACHE_PAGE_TIMEOUT', 300 );
\define( 'SUITE_UNIQUE_KEY', '' );
\define( 'OUTPUT_CACHE_METHOD', 'Redis' );

 

[Martin A.]

Since both your store method and cache method is the same, not a whole lot will be stored as a "cache". The cache is mostly used for things that expire, and data store for things that should persist for a longer period. But if your store method is set to "File System" (for instance), and you have defined a different cache method (Redis, Memcache, XCache, .... ), it will store it in both.

Since you use Redis for both, there is no point to place the data in both the data store and cache store. You will only see the things that are specifically stored in the cache. What you see in the image is perfectly normal.

(This is just my impression of this from quickly looking at the code)

[ABGenc]

1 minute ago, Martin A. said:

But if your store method is set to "File System" (for instance), and you have defined a different cache method (Redis, Memcache, XCache, .... ), it will store it in both.

When you set store method to File System and cache to Redis, store option is also set to Redis and exactly the same config applies. I have just tried and seen that.

[Martin A.]

8 hours ago, ABGenc said:

When you set store method to File System and cache to Redis, store option is also set to Redis and exactly the same config applies. I have just tried and seen that.

That makes sense, as Redis can handle both things. This only applies to the other cache options then.

[AlexJ]

On 9/12/2019 at 4:42 PM, ABGenc said:

\define( 'SUITE_UNIQUE_KEY', '' );

Why it's null for you? For me it shows some value.. may be that's causing issue for u? 

[ABGenc]

7 hours ago, AlexJ said:

Why it's null for you? For me it shows some value.. may be that's causing issue for u? 

Sorry for confusing. Actually there is a value but I have deleted it before posting .