Beta 8 - Reduced performance mobile

[Matt]

As long as you pass code web vitals, you’ll be fine.

Also good luck running a large busy site on SQLite. 😀

[Adlago]

I still believe that this wish of mine from 2021 will start with the release of version 5.

 

[AlexJ]

On 12/18/2024 at 7:50 AM, Dll said:

So, setting it for one hour will make it faster across the hour than setting it for 5 minutes. 

How does this caching work? It's event or scheduled based to pull latest topic/post? For Ex: Caching is at 15 mins, it will only generate the block data every 15 mins OR Topics / Post are event based and will update the store data. Then user uses gets that as a static block, so request wouldn't be dynamically processed for each user?

I am not sure if I am explaining my question correctly. I am hoping it's event driven block update and then user refreshes the browser, request wouldn't go to database rather cache. 

[Jim M]

15 hours ago, AlexJ said:

How does this caching work? It's event or scheduled based to pull latest topic/post? For Ex: Caching is at 15 mins, it will only generate the block data every 15 mins OR Topics / Post are event based and will update the store data. Then user uses gets that as a static block, so request wouldn't be dynamically processed for each user?

I am not sure if I am explaining my question correctly. I am hoping it's event driven block update and then user refreshes the browser, request wouldn't go to database rather cache. 

Blocks are cached when a user views it for the duration set. There is no background tasks updating data based on that timeframe.

[KT Walrus]

On 12/31/2024 at 2:44 PM, Matt said:

As long as you pass code web vitals, you’ll be fine.

Also good luck running a large busy site on SQLite. 😀

I just finished paying one of the top Laravel agencies over $100k to prototype this app in Laravel using SQLite (with me on Zoom pairing with the lead programmer for the last 2 months on exploring all my ideas for SQLite at scale) and I am convinced SQLite will scale quite nicely for a large busy site.

I am now hoping to apply the knowledge I gained over the last 5 months to an IC5 app using SQLite. I got tired of waiting for Invision to release IC5 so I spent some money on prototyping my ideas in Laravel.

Checkout Turso/LibSQL and also Fly.io for more on making SQLite scale for large busy sites. 

I prototyped LibSQL but ended up going with standard SQLite (with using the newly released sqlite3_rsync tool for syncing to UI servers instead of LibSQL’s embedded replicas). 

I believe Invision could absolutely replace MySQL with SQLite for Cloud customers and cut your database expenses to a fraction of what you spend with AWS on MySQL and end up with extremely fast queries as a bonus. SQLite queries execute in nanoseconds while MySQL queries take milliseconds. 
 

[Matt]

On 1/2/2025 at 3:09 AM, KT Walrus said:

I just finished paying one of the top Laravel agencies over $100k to prototype this app in Laravel using SQLite (with me on Zoom pairing with the lead programmer for the last 2 months on exploring all my ideas for SQLite at scale) and I am convinced SQLite will scale quite nicely for a large busy site.

I am now hoping to apply the knowledge I gained over the last 5 months to an IC5 app using SQLite. I got tired of waiting for Invision to release IC5 so I spent some money on prototyping my ideas in Laravel.

Checkout Turso/LibSQL and also Fly.io for more on making SQLite scale for large busy sites. 

I prototyped LibSQL but ended up going with standard SQLite (with using the newly released sqlite3_rsync tool for syncing to UI servers instead of LibSQL’s embedded replicas). 

I believe Invision could absolutely replace MySQL with SQLite for Cloud customers and cut your database expenses to a fraction of what you spend with AWS on MySQL and end up with extremely fast queries as a bonus. SQLite queries execute in nanoseconds while MySQL queries take milliseconds. 
 

We pay for storage, not transactions so it won't really affect costings. I'm not sure that switching to SQLite (at considerable cost) will actually make any tangible benefit in terms of speed.

[KT Walrus]

2 hours ago, Matt said:

We pay for storage, not transactions so it won't really affect costings. I'm not sure that switching to SQLite (at considerable cost) will actually make any tangible benefit in terms of speed.

You could start with SQLite for caching data locally on the server. If settings, compiled theme templates, etc. were stored in SQLite, all those queries that are done on every page load would be lightening fast (even faster than fopen). Because SQLite PDO driver is built into PHP and creating a SQLite database is simply touching a file on the server you don’t really need any admin settings to deploy (you should connect to the database in WAL mode though).

You could also use SQLite for the Database or Filesystem Storage Methods for files as a fast, easy to use database (as opposed to the local file system). 

Backing up SQLite is simply copying the database file (using sqlite3_rsync tool if database is live).

Also, SQLite would be a better alternative for sessions than Redis (for installations on a single server)  

Everything about using SQLite is so easy and so fast. In my Laravel prototype, I was able to eliminate aggregate data about content (like stored in the forums table) to show this data like unread counts, last replier, etc. 

When you aren’t worried about doing 200 queries per page load, you can really simplify the SQL that you write to get the data for showing to a user and not worry about caching it for subsequent page loads.

So, even if you don’t want to support storing everything in SQLite, you could make use SQLite to really eliminate many network round trips to the MySQL instance in the Invision Cloud. I really think you could make a difference that your customers would appreciate especially us self-hosting customers. 

[EliasM]

8 minutes ago, KT Walrus said:

You could start with SQLite for caching data locally on the server. If settings, compiled theme templates, etc. were stored in SQLite, all those queries that are done on every page load would be lightening fast (even faster than fopen). Because SQLite PDO driver is built into PHP and creating a SQLite database is simply touching a file on the server you don’t really need any admin settings to deploy (you should connect to the database in WAL mode though).

You could also use SQLite for the Database or Filesystem Storage Methods for files as a fast, easy to use database (as opposed to the local file system). 

Backing up SQLite is simply copying the database file (using sqlite3_rsync tool if database is live).

Also, SQLite would be a better alternative for sessions than Redis (for installations on a single server)  

Everything about using SQLite is so easy and so fast. In my Laravel prototype, I was able to eliminate aggregate data about content (like stored in the forums table) to show this data like unread counts, last replier, etc. 

When you aren’t worried about doing 200 queries per page load, you can really simplify the SQL that you write to get the data for showing to a user and not worry about caching it for subsequent page loads.

So, even if you don’t want to support storing everything in SQLite, you could make use SQLite to really eliminate many network round trips to the MySQL instance in the Invision Cloud. I really think you could make a difference that your customers would appreciate especially us self-hosting customers. 

So do you have a site that uses SQLite so that we check the speed of your site?

Edited Friday at 12:28 PM by EliasM

[KT Walrus]

23 minutes ago, EliasM said:

So do you have a site that uses SQLite so that we check the speed of your site?

No. I haven’t coded my Invision app yet as I’m waiting for IC5 GA. 
 

I have contracted with Tighten to develop a Laravel app for the backend of my new IC5 app though. This should be finished in the next few weeks. The backend will manage the SQLite databases. Each user that visits my site will have the backend generate the SQLite database for just that session. Dynamically generating these SQLite databases are lightning fast and contain only the data the user has permission to see. 
 

You can check out this site that uses Laravel and SQLite in the meantime:

https://pinkary.com

 

[EliasM]

4 minutes ago, KT Walrus said:

No. I haven’t coded my Invision app yet as I’m waiting for IC5 GA. 
 

I have contracted with Tighten to develop a Laravel app for the backend of my new IC5 app though. This should be finished in the next few weeks. The backend will manage the SQLite databases. Each user that visits my site will have the backend generate the SQLite database for just that session. Dynamically generating these SQLite databases are lightning fast and contain only the data the user has permission to see. 
 

You can check out this site that uses Laravel and SQLite in the meantime:

https://pinkary.com

 

We will not be able to compare the speed based on another script, where programming is a key factor in increasing the speed, so it does not apply to databases, so in order to be able to determine, the speed must be compared on IC5

[Adlago]

@Matt 

A few 4.x. versions ago there was a "cache page for guests ", which created a lot of issues, vanity especially in such tests.
Now, after almost a hundred more tests in a few days - I come to a conclusion - in version 5 you have included such a cache page for guests, and without an option to disable it from the administrator.
Why I think so:
1. Here is a PSI server that tests my site

2. First byte is almost always 180 to 220 milliseconds - and then a result of 69-70 performance points.
3. I created an html file from the source code of my test installation
PSI test is identical 69-70 performance points and 180-220 milliseconds first byte...
Conclusion - version 5 is not loaded directly when requested by a browser, but a preliminary html file is loaded, which is why the LCP is delayed a lot until the CSS framework is downloaded (PSI tests with an introduced 4G delay for mobile)

When I hit a good result in a test, the first byte is already within the normal range of 350-500 milliseconds, LCP is 2.2 seconds and the performance is over 90 points. Yes, with multiple tests and this moment I manage to hit - or this is the moment after refreshing the cache on the guest page.
I have many screenshots of the different states, but there is no point in posting them.
Here are just two - on my test site and the second in the html file that I created from the source code....
Please review your code and remove this guest cache page - or at least make it an on/off option in ACP.

 

[AlexJ]

On 1/1/2025 at 8:49 AM, Jim M said:

Blocks are cached when a user views it for the duration set. There is no background tasks updating data based on that timeframe.

Thanks for the confirmation. Can it be modified, so that it's event based?

If event driven is not possible, schedule would help as well i.e. every 5 mins run 1 query, update block, static block (5 mins)  -> all users get it if it's configured as don't check for permissions. 

 

[Hatsu]

I can imagine doing this event based circumvents the reason for caching. If you have a small forum and less traffic it doesn't matter. But on a large forum you would perhaps increase the number of queries more than without caching (depending on the block).

[Adlago]

Another paradox - this only happens with version 5. Never with version 4.x

and of course - the lowest level of performance

Edited Saturday at 11:15 AM by Adlago

[AlexJ]

14 hours ago, Hatsu said:

I can imagine doing this event based circumvents the reason for caching. If you have a small forum and less traffic it doesn't matter. But on a large forum you would perhaps increase the number of queries more than without caching (depending on the block).

It helps with larger forum as well. Imagine you have 500 users browsing and each having it's own request i.e. 500 DB request.

Now with event/schedule based, 1 query every 30 sec (OR configurable) -> Update block cache -> Server all 500 users that without any database request. 

I am not sure if MySQL has build in cache i.e. if content hasn't change, does it still do full DB scan OR provides from cache. I know for sure, snowflake has it's caching engine. So if my underlying data hasn't change my next query execution comes from cache.

[Matt]

Our Cloud platform uses Redis for data caching (data stores, temporary caches, etc) and CloudFlare for CDN caching (mostly for guests).

23 hours ago, Adlago said:

Conclusion - version 5 is not loaded directly when requested by a browser, but a preliminary html file is loaded, which is why the LCP is delayed a lot until the CSS framework is downloaded (PSI tests with an introduced 4G delay for mobile)

This does not happen. We serve up a text/html page in full for each request. As mentioned above, our Cloud platform will use Cloudflare to serve this page to guests if a suitable cached version is found, but logged in members always get a normal page.

23 hours ago, Adlago said:

Please review your code and remove this guest cache page - or at least make it an on/off option in ACP.

Again, this not a feature.

25 minutes ago, AlexJ said:

It helps with larger forum as well. Imagine you have 500 users browsing and each having it's own request i.e. 500 DB request.

Now with event/schedule based, 1 query every 30 sec (OR configurable) -> Update block cache -> Server all 500 users that without any database request. 

I am not sure if MySQL has build in cache i.e. if content hasn't change, does it still do full DB scan OR provides from cache. I know for sure, snowflake has it's caching engine. So if my underlying data hasn't change my next query execution comes from cache.

MySQL does have some short term query caching. We cannot really cache anything for logged in members or you would have issues with sessions, CSRF tokens, content on the page you can't see, etc, etc.

[AlexJ]

6 minutes ago, Matt said:

Our Cloud platform uses Redis for data caching (data stores, temporary caches, etc) and CloudFlare for CDN caching (mostly for guests).

How does Redis performs? Asking because I never see Redis using more then 20-40Mb max. So is it really that optimized or it's not evening caching correctly? My database size is around 2.5G appx.

6 minutes ago, Matt said:

MySQL does have some short term query caching. We cannot really cache anything for logged in members or you would have issues with sessions, CSRF tokens, content on the page you can't see, etc, etc.

I see - thanks. 

[Matt]

I'd say that's about right. It doesn't cache full HTML page outputs. It really acts as a memory layer between the database and PHP. We put things in there that don't change very often and it saves a DB lookup.