Update to Data Storage and Caching advice?

[TSP]

I'm wondering whether you still consider the following description to be true: 

The data storage method is used by various systems to save arbitrary data. If caching is enabled, using the MySQL Database provides the best performance for most environments, otherwise using the File System is usually best.

I think a little while ago you introduced Redis as an option for the Data Storage part, so what are your experiences here and should the description maybe be updated? 

 

[Marc Stridgen]

That description would still be true for most environments. Most environments will not need redis.

[TSP]

6 minutes ago, Marc Stridgen said:

That description would still be true for most environments. Most environments will not need redis.

Okay, so which environments would you use Redis for? 

Besides, if Redis is already used for the caching, then I guess it would be best to use it for Data Storage too? 

[Marc Stridgen]

On 11/26/2021 at 4:10 PM, TSP said:

Okay, so which environments would you use Redis for? 

Besides, if Redis is already used for the caching, then I guess it would be best to use it for Data Storage too? 

It may be best to use it for data storage, yes. Generally redis would only be used for very large and busy sites, and whereby the redis instance is stored externally to your server environment, in order to offload.

[Matt]

It's a complicated question to answer without seeing more site specifics.

Generally speaking, MySQL is fine. The queries are fast and lightweight so shouldn't cause any real increase in load.

If you simply install Redis on the same server as MySQL then you won't gain a huge amount (perhaps a little speed but nothing earth shattering).

If you have a busy site, then Redis is a good option as it offloads I/O to MySQL which can be beneficial at scale. In that case, then definitely use Redis for both caching and data storage with the caveat that you need to manage server resources to ensure giving Redis RAM doesn't starve MySQL or http.

[marklcfc]

I've used file system majority of the time, always seemed fast to me. I've often wondered if a file system on a ssd drive is better than the other options though which is partly why I've stuck with it.

Edited November 29, 2021 by marklcfc

[Matt]

If you want to get really technical, there's this: https://konstantin.blog/2021/redis-vs-memcached-vs-file_get_contents/

We use Redis on our infrastructure because it's a great object cache and as we have thousands of connections to MySQL a second, it helps reduce load there. Redis is its own instance, and not taking resources from a single server.

[Terry Ellison]

@Matt I can see to clear advantages of caching for complex apps such as yours, MediaWiki, WordPress, etc. as this typically gives a 2× or better throughput improvement.  I can also see the advantages from your PoV as a vendor in dropping APCu, memcache, etc. for a single supported cache: Redis.

What confuses me is that selecting Redis as a cache seems to state that use of Redis for forces a swap from MySQL DB as the primary data store: ACP->Advanced Configuration->Data Storage Method is switched from MySQL to Redis.  If so this has huge data security / integrity / backup implications. If not, then this menu option needs rewording to clarify.  Adding some decent configuration documentation would also help.

ATM, all I can do is to retro-engineer the implications on a test instance. 😒

[Terry Ellison]

To follow this point after configuring Redis, the phrase Data Storage Method is very misleading here because even with the Redis Data Storage Method ticked, MySQL is still used for the relational data store for all of the user, topics, posts , etc. related data.  Redis is only used as a memory cache for essentially ephemeral data, very much in the way that memcached was previously used.     

[Marc Stridgen]

1 hour ago, Terry Ellison said:

To follow this point after configuring Redis, the phrase Data Storage Method is very misleading here because even with the Redis Data Storage Method ticked, MySQL is still used for the relational data store for all of the user, topics, posts , etc. related data.  Redis is only used as a memory cache for essentially ephemeral data, very much in the way that memcached was previously used.     

Please feel free to post this up within our suggestions area if you believe there is something you would like to see changed