ASTRAPI

Looks like I spoke too soon, overnight it has got to 31% and appears to be stuck, I can't see anything obvious. I tried running the background processes "now" but this just returns a blank page after doing the infinite loading circle for a while. Not really sure how to diagnose it?
UPDATE: I fixed this by increasing the PHP memory limit.

Woohoo! It works, thanks so much @ASTRAPI
I have 20gb of uploads to move across now, am I okay doing this by switching the filesystem in my IPS dash? Roughly how long could it take?
 

The tutorial is for using Cloudflare also so you can benefit wit the partner agreement between Wasabi and Cloudflare for unlimited traffic between them and benefit also from the usage of Cloudflare CDN....

For the certificate you just need to select the Full option at Cloudflare....

I've made a note for us to review this area of code for 4.5.

Here's my latest update on how I got everything figured out: 
1. Utilizing the set-up I referenced above for the usawxcdn, I had to wait about 10 minutes and then Cloudflare seemed to kick in with its SSL certificate.
2. The fix re. Full vs. Strict SSL enforcement on Cloudflare did work as well. However, I had to first switch the setting off and then switch it to Full. It didn't seem to take until I did that.
With all that said, I have everything working as expected. Thanks to all for the tips in this thread, all the information together helped me to get this up and urnning. 

Yes it will be great if there is another way to do that or be able to improve the current code for this 🙂

Please check the whole logic for this.
More than 2 hours - I changed sleep from 1 second to 100 ms using usleep.
More than 95% of my first byte (waiting time) are excellent - 90-140 ms, in other cases they are from 200 to 500 ms.
Probably the logic of what you say does not work of plan.

In this topic, @ASTRAPI made a detailed description of what is and what the first byte contains, for which I thank him.
 
For a few months now, I've been doing a lot of first byte analysis and testing in my installation.
Here are the results I get - pay attention, these results are when tested with a Chrome browser developer tool and I only look at the waiting time, ie. real time in which the server started to work after request, and produces source code.
Sorted as my stats, the results (probably over 1000 test) look like this:
70% - waiting time in the interval 90 - 150 ms
20% - waiting time in the interval 150- 400 ms
8% - waiting time in the interval 400- 1000 ms
2% - waiting time in the interval over 1 second, often over 2 seconds
I created a ticket for my host company because my host plan is expensive and with good parameters guaranteed.
The guys from the host company have done more than 24 hours of analysis and here's what they found
Below I post their response.
======================
In one of the many checks that it develops, we notice that "sleep" during loading and run all that are generating source code scripts. To look at the sleep effect of the application, apply a sample of the command that checks all file calls (at system level) during site loading:
16:03:15.969084 open("/home/database/public_html/system/Session/Front.php", O_RDONLY) = 4 16:03:15.970734 open("/home/database/public_html/system/Session/Session.php", O_RDONLY) = 4 16:03:15.971749 open("/home/database/public_html/system/Theme/Theme.php", O_RDONLY) = 4 16:03:16.983086 open("/home/database/public_html/system/Node/Model.php", O_RDONLY) = 4 After executing the /home/database/public_html/system/Theme/Theme.php file, we see that it is almost a second before the next script is loaded and /home/database/public_html/system/Node/Model.php. For more precision, we see that Theme.php started its implementation at 16: 03: 15.971749, and a second later, at 16: 03: 16.983086, the Model.php implementation started. This second delay is due to the following snippet of Theme.php code:
/* If we don't have a compiled template, do that now */ if ( ! $cachedObject and !isset( \IPS\Data\Store::i()->$key ) ) { /* It can take a few seconds for templates to finish compiling if initiated elsewhere, so let's try a few times sleeping 1 second between attempts to give the compilation time to finish */ $attempts = 0; while( $attempts < 6 ) { if ( $attempts === 5 ) { /* Rebuild in progress */ \IPS\Log::debug( "Template store key: {$key} rebuilding and requested again ({$app}, {$location}, {$group})", "template_store_building" ); /* Since we can't do anything else, this ends up just being an uncaught exception - show the error page right away to avoid the unnecessary logging */ \IPS\IPS::genericExceptionPage(); } $built = $this->compileTemplates( $app, $location, $group ); if ( $built === NULL ) { $attempts++; sleep(1); } else { break; } } /* Still no key? */ if ( ! isset( \IPS\Data\Store::i()->$key ) ) { \IPS\Log::log( "Template store key: {$key} missing ({$app}, {$location}, {$group})", "template_store_missing" ); throw new \ErrorException( 'template_store_missing' ); } } This part of the code checks whether a compiled template is already available and if not, compiles. Several attempts are made in the compilation process, which according to the code comment may take a few seconds. In our opinion, the presence of sleep (1) is the specific reason for the delay sometimes occurring.
For our part, we might recommend that you consult the developers who assisted with the site's construction. From our many attempts to reproduce the delay, we see that to a large extent the site is well optimized, but rebuilding templates is a functionality that needs to be discussed with developers.
In our view, this part of the code is the reason for the TTFB indicator's unconscious state, since it directly depends on the time it takes to process all the functionality involved in generating the source code.
==================================
Whether these guys from the analysis department of my host company are right - I don't know.
I share it here and look forward to clarification.
Thanks.
 

Due to recent rapid growth, we are currently experiencing unforeseen data throughput capacity issues in our us-east-1 data center.
In order to meet the needs of our customers, we are deploying an additional us-east data center (us-east-2) that is planned for operation by 1 October 2019.  
We are also taking steps to improve throughput capacity in us-east-1.  
We do have available throughput capacity in our us-west and eu-central data centers (if those locations are options for your storage use case).  

One of the best and free that you can use is Cloudflare as it can take care your DNS and static files like images .... and it can offer also some other optimizations !
They have almost everywhere a datacenter...
https://www.cloudflare.com/network/
 

Just edit my topic with a note about that Invision is using something specific (header?) to S3 and only Downloads can't work with Wasabi.
Any related info? It will be great if that can be adjusted and work on both so users can benefit on both cases...
Maybe @bfarber ?
Checking also this one:
 

Anything legal should be fine 🙂
If you are not sure about your content then ask them and see ....
 

You are welcome 🙂
I am getting a lot of positive feedback about this topic and i think (if admins agree) it will be helpfull to make it a pinned topic ....?

At Cloudflare on Crypto section select Full and check if it works.
If not then check again the tutorial step by step....
There is an issue with your certificate at your server:
Error 526: Invalid SSL certificate
Error 526 happens when Cloudflare is unable to successfully validate the SSL certificate on the origin web server and the SSL setting in the Cloudflare Crypto app is set to Full SSL (Strict) for the website.  
https://support.cloudflare.com/hc/en-us/articles/200721975-Error-526-Invalid-SSL-certificate
 

Yes you will need to add your card and charged 5,99$ per month for 1TB storage and unlimited bandwidth to get it work with Invision.
* I did the related edit using Bold on my topic.
* I add also an example of a Cloudflare Cname record.
*I add also a notice that is better to not move to Wasabi the theme resources... For all the rest is ok.

Assuming from your settings that you are using the European datacenter:
Please double check all settings from the tutorial and most important the DNS part:
cname -> cdn -> cdn.yourdomain.com.s3.eu-central-1.wasabisys.com  <--For Europe datacenter
or
cname -> cdn -> cdn.yourdomain.com.s3.us-west-1.wasabisys.com  <--For US-WEST datacenter
or
cname -> cdn -> cdn.yourdomain.com.s3.wasabisys.com  <--For US-EAST datacenter
 
So your record should be: 
cname -> cdn -> cdn.mysite.ca.s3.wasabisys.com
 
Then give it a minute and try again....
 
I just edited my topic to clarify that entry ....

You can use it on any environment that supports natively or by any plugin/addon Amazon S3. For CiC (community in cloud) yes it will work!
You can select from your admin CP which files you like to move and which not 🙂

Hello
(This is a community guide on how to setup Wasabi and Cloudflare, as a significantly cheaper replacement for Amazon S3 and Cloudfront.  The author has no affiliate arrangement with any of the companies listed in the guide and offers the guide as a free community resource.  You should consult with your hosting provider and / or server administrator before proceeding with the guide.)
BACKGROUND:
Invision Communities contain rich content, from forum attachments to gallery images to download files.  These resources add up over time. 
Here are real examples of communities that I provide server administration for:
- Website A: 400 GB of total storage and 1.2 TB of total monthly bandwidth 
- Website B: 800 GB of total storage and 14 TB of total bandwidth
Invision Community allows a file storage configuration with Amazon S3 and a content distribution network (CDN) of your choice:
 
WHAT IS WASABI?
Wasabi Cloud  is faster and cheaper than Amazon S3 for object storage, offers unlimited egress for bandwidth, and compatible with Cloudflare.  One of the biggest hidden expenses of using Amazon S3 is not the cost of storage itself, but the bandwidth to deliver objects in and out.  Wasabi does not charge for egress bandwidth, which can save on the majority of the true cost of Amazon S3. It also provides an S3-compliant interface, which means it can be used as a perfect drop-in replacement for Amazon S3 storage and immediately usable on all Invision Communities.   

Wasabi is 80% cheaper and 6x faster than Amazon S3, with 100% data immutability protection, and no data egress fees or API charges. Created by Carbonite co-founders and cloud storage pioneers David Friend and Jeff Flowers, Wasabi is on a mission to commoditize the storage industry. Wasabi is a privately held company based in Boston, MA.
In March of 2018, Wasabi pioneered the cloud storage industry’s first ‘unlimited free egress’ pricing plan. We are excited to join forces with Cloudflare and the Bandwidth Alliance, validating that the early business models of Cloud 1.0 vendors no longer applies in a world that expects unlimited everything.  What this means in practice is that you can use Cloudflare services (including the free plan) like blazing-fast DNS, WAF firewall, performance improvements on the fly, and much more.  Forums users will request a topic, and all static content can be picked up from Cloudflare's optimized services that will connect to Wasabi.

 
At the moment, Wasabi offers three Datacenters:
1)US-WEST
2)US-EAST
3)EU-CENTRAL 

One price. Lightning fast speed. No hidden fees.

You can check and compare the prices above.
 
GUIDE:
BEFORE PROCEEDING, PLEASE BACKUP YOUR DATA FIRST!
YOUR FILES WILL BE IN PRIVATE MODE AND WILL NOT BE ACCESSIBLE ON YOUR FORUM !
YOU MUST GET A PAID PLAN TO BE ABLE TO ACTIVATE THE INVISION FILE PERMISSIONS AND GET YOUR FILES ACCESSIBLE AT YOUR FORUMS.
JUST ADD YOUR CREDIT CARD NOW (AT YOUR PROFILE PAGE) AND THEY WILL CHARGE YOU 5,99$ FOR A MONTH FOR 1TB STORAGE AND UNLIMITED BANDWIDTH.
THE FREE TRIAL WILL NOT WORK FOR INVISION !
 
Step 1: Create a CName at CloudFlare
We start by creating a cname at Cloudflare:
cname -> cdn -> cdn.yourdomain.com.s3.eu-central-1.wasabisys.com  <--For Europe datacenter
or
cname -> cdn -> cdn.yourdomain.com.s3.us-west-1.wasabisys.com  <--For US-WEST datacenter
or
cname -> cdn -> cdn.yourdomain.com.s3.wasabisys.com  <--For US-EAST datacenter
Like this:

 
Step 2: Create a bucket on Wasabi
First we sign up for a free account at Wasabi Cloud: https://wasabi.com/sign-up/
Then we create a bucket:

 
It is important to name it as: cdn.yourdomain.com
 

 
Then keep default suspend Versioning and Suspend Logging:

Then click CREATE BUCKET:
 

 
Step 3: Retrieve your Access Keys
Go to the menu on top left and click Access keys on the left. Click on "Create New Access Key."

 
Copy / save the access key and the secret key:

Step 4: Setting up a new file storage in ACP
Login to Admin Control Panel and navigate to Overview at the Files category and then click at the right side at Storage settings:

 
Then we click at Configurations and at the right side at Create New:
 

We select Amazon s3 and we add the following info:

Save it.
Step 5: Set the new storage at storage settings
Navigate to Storage settings and select from the drop down menu the new Amazon s3 bucket.

 
Keep only at your server your theme resources:

*Attachments and Downloads doesn't work out of the box as it seems there is some hardcoded code at Invision related to S3...
To make them work you just need to install the addon bellow: (tested with Invision 4.4.4).
 
Then wait for any Invision background tasks to finish and check.
I hope that you will benefit from this tutorial and get better performance while saving a lot of money 🙂
Enjoy!
OTHER NOTES:
If you are already using S3 then it is very easy to move straight to Wasabi by following the above tutorial as it is NOT needed to move back your files to server and then to Wasabi.
CREDITS
Credits to @Joel R for giving me the motivation to write this tutorial and providing screenshots from his Wasabi account and for editing the topic.
Thank you

Yes it works.
Please check again my instructions on first page...

Makes sense - thank you for clarifying!

Hello
(This is a community guide on how to setup Wasabi and Cloudflare, as a significantly cheaper replacement for Amazon S3 and Cloudfront.  The author has no affiliate arrangement with any of the companies listed in the guide and offers the guide as a free community resource.  You should consult with your hosting provider and / or server administrator before proceeding with the guide.)
BACKGROUND:
Invision Communities contain rich content, from forum attachments to gallery images to download files.  These resources add up over time. 
Here are real examples of communities that I provide server administration for:
- Website A: 400 GB of total storage and 1.2 TB of total monthly bandwidth 
- Website B: 800 GB of total storage and 14 TB of total bandwidth
Invision Community allows a file storage configuration with Amazon S3 and a content distribution network (CDN) of your choice:
 
WHAT IS WASABI?
Wasabi Cloud  is faster and cheaper than Amazon S3 for object storage, offers unlimited egress for bandwidth, and compatible with Cloudflare.  One of the biggest hidden expenses of using Amazon S3 is not the cost of storage itself, but the bandwidth to deliver objects in and out.  Wasabi does not charge for egress bandwidth, which can save on the majority of the true cost of Amazon S3. It also provides an S3-compliant interface, which means it can be used as a perfect drop-in replacement for Amazon S3 storage and immediately usable on all Invision Communities.   

Wasabi is 80% cheaper and 6x faster than Amazon S3, with 100% data immutability protection, and no data egress fees or API charges. Created by Carbonite co-founders and cloud storage pioneers David Friend and Jeff Flowers, Wasabi is on a mission to commoditize the storage industry. Wasabi is a privately held company based in Boston, MA.
In March of 2018, Wasabi pioneered the cloud storage industry’s first ‘unlimited free egress’ pricing plan. We are excited to join forces with Cloudflare and the Bandwidth Alliance, validating that the early business models of Cloud 1.0 vendors no longer applies in a world that expects unlimited everything.  What this means in practice is that you can use Cloudflare services (including the free plan) like blazing-fast DNS, WAF firewall, performance improvements on the fly, and much more.  Forums users will request a topic, and all static content can be picked up from Cloudflare's optimized services that will connect to Wasabi.

 
At the moment, Wasabi offers three Datacenters:
1)US-WEST
2)US-EAST
3)EU-CENTRAL 

One price. Lightning fast speed. No hidden fees.

You can check and compare the prices above.
 
GUIDE:
BEFORE PROCEEDING, PLEASE BACKUP YOUR DATA FIRST!
YOUR FILES WILL BE IN PRIVATE MODE AND WILL NOT BE ACCESSIBLE ON YOUR FORUM !
YOU MUST GET A PAID PLAN TO BE ABLE TO ACTIVATE THE INVISION FILE PERMISSIONS AND GET YOUR FILES ACCESSIBLE AT YOUR FORUMS.
JUST ADD YOUR CREDIT CARD NOW (AT YOUR PROFILE PAGE) AND THEY WILL CHARGE YOU 5,99$ FOR A MONTH FOR 1TB STORAGE AND UNLIMITED BANDWIDTH.
THE FREE TRIAL WILL NOT WORK FOR INVISION !
 
Step 1: Create a CName at CloudFlare
We start by creating a cname at Cloudflare:
cname -> cdn -> cdn.yourdomain.com.s3.eu-central-1.wasabisys.com  <--For Europe datacenter
or
cname -> cdn -> cdn.yourdomain.com.s3.us-west-1.wasabisys.com  <--For US-WEST datacenter
or
cname -> cdn -> cdn.yourdomain.com.s3.wasabisys.com  <--For US-EAST datacenter
Like this:

 
Step 2: Create a bucket on Wasabi
First we sign up for a free account at Wasabi Cloud: https://wasabi.com/sign-up/
Then we create a bucket:

 
It is important to name it as: cdn.yourdomain.com
 

 
Then keep default suspend Versioning and Suspend Logging:

Then click CREATE BUCKET:
 

 
Step 3: Retrieve your Access Keys
Go to the menu on top left and click Access keys on the left. Click on "Create New Access Key."

 
Copy / save the access key and the secret key:

Step 4: Setting up a new file storage in ACP
Login to Admin Control Panel and navigate to Overview at the Files category and then click at the right side at Storage settings:

 
Then we click at Configurations and at the right side at Create New:
 

We select Amazon s3 and we add the following info:

Save it.
Step 5: Set the new storage at storage settings
Navigate to Storage settings and select from the drop down menu the new Amazon s3 bucket.

 
Keep only at your server your theme resources:

*Attachments and Downloads doesn't work out of the box as it seems there is some hardcoded code at Invision related to S3...
To make them work you just need to install the addon bellow: (tested with Invision 4.4.4).
 
Then wait for any Invision background tasks to finish and check.
I hope that you will benefit from this tutorial and get better performance while saving a lot of money 🙂
Enjoy!
OTHER NOTES:
If you are already using S3 then it is very easy to move straight to Wasabi by following the above tutorial as it is NOT needed to move back your files to server and then to Wasabi.
CREDITS
Credits to @Joel R for giving me the motivation to write this tutorial and providing screenshots from his Wasabi account and for editing the topic.
Thank you

Check at Wasabi panel if there is any setting related to make the bucket files/folder public and enable it and check again.....
I am out now and i can’t check it sorry ....

Once again though, this should be pinned. (Spoke with wasabi, they'll allow our content, wooo!). Very very useful to anyone using S3.

Hello
(This is a community guide on how to setup Wasabi and Cloudflare, as a significantly cheaper replacement for Amazon S3 and Cloudfront.  The author has no affiliate arrangement with any of the companies listed in the guide and offers the guide as a free community resource.  You should consult with your hosting provider and / or server administrator before proceeding with the guide.)
BACKGROUND:
Invision Communities contain rich content, from forum attachments to gallery images to download files.  These resources add up over time. 
Here are real examples of communities that I provide server administration for:
- Website A: 400 GB of total storage and 1.2 TB of total monthly bandwidth 
- Website B: 800 GB of total storage and 14 TB of total bandwidth
Invision Community allows a file storage configuration with Amazon S3 and a content distribution network (CDN) of your choice:
 
WHAT IS WASABI?
Wasabi Cloud  is faster and cheaper than Amazon S3 for object storage, offers unlimited egress for bandwidth, and compatible with Cloudflare.  One of the biggest hidden expenses of using Amazon S3 is not the cost of storage itself, but the bandwidth to deliver objects in and out.  Wasabi does not charge for egress bandwidth, which can save on the majority of the true cost of Amazon S3. It also provides an S3-compliant interface, which means it can be used as a perfect drop-in replacement for Amazon S3 storage and immediately usable on all Invision Communities.   

Wasabi is 80% cheaper and 6x faster than Amazon S3, with 100% data immutability protection, and no data egress fees or API charges. Created by Carbonite co-founders and cloud storage pioneers David Friend and Jeff Flowers, Wasabi is on a mission to commoditize the storage industry. Wasabi is a privately held company based in Boston, MA.
In March of 2018, Wasabi pioneered the cloud storage industry’s first ‘unlimited free egress’ pricing plan. We are excited to join forces with Cloudflare and the Bandwidth Alliance, validating that the early business models of Cloud 1.0 vendors no longer applies in a world that expects unlimited everything.  What this means in practice is that you can use Cloudflare services (including the free plan) like blazing-fast DNS, WAF firewall, performance improvements on the fly, and much more.  Forums users will request a topic, and all static content can be picked up from Cloudflare's optimized services that will connect to Wasabi.

 
At the moment, Wasabi offers three Datacenters:
1)US-WEST
2)US-EAST
3)EU-CENTRAL 

One price. Lightning fast speed. No hidden fees.

You can check and compare the prices above.
 
GUIDE:
BEFORE PROCEEDING, PLEASE BACKUP YOUR DATA FIRST!
YOUR FILES WILL BE IN PRIVATE MODE AND WILL NOT BE ACCESSIBLE ON YOUR FORUM !
YOU MUST GET A PAID PLAN TO BE ABLE TO ACTIVATE THE INVISION FILE PERMISSIONS AND GET YOUR FILES ACCESSIBLE AT YOUR FORUMS.
JUST ADD YOUR CREDIT CARD NOW (AT YOUR PROFILE PAGE) AND THEY WILL CHARGE YOU 5,99$ FOR A MONTH FOR 1TB STORAGE AND UNLIMITED BANDWIDTH.
THE FREE TRIAL WILL NOT WORK FOR INVISION !
 
Step 1: Create a CName at CloudFlare
We start by creating a cname at Cloudflare:
cname -> cdn -> cdn.yourdomain.com.s3.eu-central-1.wasabisys.com  <--For Europe datacenter
or
cname -> cdn -> cdn.yourdomain.com.s3.us-west-1.wasabisys.com  <--For US-WEST datacenter
or
cname -> cdn -> cdn.yourdomain.com.s3.wasabisys.com  <--For US-EAST datacenter
Like this:

 
Step 2: Create a bucket on Wasabi
First we sign up for a free account at Wasabi Cloud: https://wasabi.com/sign-up/
Then we create a bucket:

 
It is important to name it as: cdn.yourdomain.com
 

 
Then keep default suspend Versioning and Suspend Logging:

Then click CREATE BUCKET:
 

 
Step 3: Retrieve your Access Keys
Go to the menu on top left and click Access keys on the left. Click on "Create New Access Key."

 
Copy / save the access key and the secret key:

Step 4: Setting up a new file storage in ACP
Login to Admin Control Panel and navigate to Overview at the Files category and then click at the right side at Storage settings:

 
Then we click at Configurations and at the right side at Create New:
 

We select Amazon s3 and we add the following info:

Save it.
Step 5: Set the new storage at storage settings
Navigate to Storage settings and select from the drop down menu the new Amazon s3 bucket.

 
Keep only at your server your theme resources:

*Attachments and Downloads doesn't work out of the box as it seems there is some hardcoded code at Invision related to S3...
To make them work you just need to install the addon bellow: (tested with Invision 4.4.4).
 
Then wait for any Invision background tasks to finish and check.
I hope that you will benefit from this tutorial and get better performance while saving a lot of money 🙂
Enjoy!
OTHER NOTES:
If you are already using S3 then it is very easy to move straight to Wasabi by following the above tutorial as it is NOT needed to move back your files to server and then to Wasabi.
CREDITS
Credits to @Joel R for giving me the motivation to write this tutorial and providing screenshots from his Wasabi account and for editing the topic.
Thank you