Community Guide on Setting Up Wasabi and CloudFlare

[ASTRAPI]

Hello

(This is a community guide on how to setup Wasabi and Cloudflare, as a significantly cheaper replacement for Amazon S3 and Cloudfront.  The author has no affiliate arrangement with any of the companies listed in the guide and offers the guide as a free community resource.  You should consult with your hosting provider and / or server administrator before proceeding with the guide.)

BACKGROUND:

Invision Communities contain rich content, from forum attachments to gallery images to download files.  These resources add up over time. 

Here are real examples of communities that I provide server administration for:

- Website A: 400 GB of total storage and 1.2 TB of total monthly bandwidth 

- Website B: 800 GB of total storage and 14 TB of total bandwidth

Invision Community allows a file storage configuration with Amazon S3 and a content distribution network (CDN) of your choice:

 

WHAT IS WASABI?

Wasabi Cloud  is faster and cheaper than Amazon S3 for object storage, offers unlimited egress for bandwidth, and compatible with Cloudflare.  One of the biggest hidden expenses of using Amazon S3 is not the cost of storage itself, but the bandwidth to deliver objects in and out.  Wasabi does not charge for egress bandwidth, which can save on the majority of the true cost of Amazon S3. It also provides an S3-compliant interface, which means it can be used as a perfect drop-in replacement for Amazon S3 storage and immediately usable on all Invision Communities.   

Wasabi is 80% cheaper and 6x faster than Amazon S3, with 100% data immutability protection, and no data egress fees or API charges. Created by Carbonite co-founders and cloud storage pioneers David Friend and Jeff Flowers, Wasabi is on a mission to commoditize the storage industry. Wasabi is a privately held company based in Boston, MA.

In March of 2018, Wasabi pioneered the cloud storage industry’s first ‘unlimited free egress’ pricing plan. We are excited to join forces with Cloudflare and the Bandwidth Alliance, validating that the early business models of Cloud 1.0 vendors no longer applies in a world that expects unlimited everything.  What this means in practice is that you can use Cloudflare services (including the free plan) like blazing-fast DNS, WAF firewall, performance improvements on the fly, and much more.  Forums users will request a topic, and all static content can be picked up from Cloudflare's optimized services that will connect to Wasabi.

 

At the moment, Wasabi offers three Datacenters:

1)US-WEST

2)US-EAST

3)EU-CENTRAL 

One price. Lightning fast speed. No hidden fees.

You can check and compare the prices above.

 

GUIDE:

BEFORE PROCEEDING, PLEASE BACKUP YOUR DATA FIRST!

YOUR FILES WILL BE IN PRIVATE MODE AND WILL NOT BE ACCESSIBLE ON YOUR FORUM !

YOU MUST GET A PAID PLAN TO BE ABLE TO ACTIVATE THE INVISION FILE PERMISSIONS AND GET YOUR FILES ACCESSIBLE AT YOUR FORUMS.

JUST ADD YOUR CREDIT CARD NOW (AT YOUR PROFILE PAGE) AND THEY WILL CHARGE YOU 5,99$ FOR A MONTH FOR 1TB STORAGE AND UNLIMITED BANDWIDTH.

THE FREE TRIAL WILL NOT WORK FOR INVISION !

 

Step 1: Create a CName at CloudFlare

We start by creating a cname at Cloudflare:

cname -> cdn -> cdn.yourdomain.com.s3.eu-central-1.wasabisys.com  <--For Europe datacenter

or

cname -> cdn -> cdn.yourdomain.com.s3.us-west-1.wasabisys.com  <--For US-WEST datacenter

or

cname -> cdn -> cdn.yourdomain.com.s3.wasabisys.com  <--For US-EAST datacenter

Like this:

 

Step 2: Create a bucket on Wasabi

First we sign up for a free account at Wasabi Cloud: https://wasabi.com/sign-up/

Then we create a bucket:

 

It is important to name it as: cdn.yourdomain.com

 

 

Then keep default suspend Versioning and Suspend Logging:

Then click CREATE BUCKET:

 

 

Step 3: Retrieve your Access Keys

Go to the menu on top left and click Access keys on the left. Click on "Create New Access Key."

 

Copy / save the access key and the secret key:

Step 4: Setting up a new file storage in ACP

Login to Admin Control Panel and navigate to Overview at the Files category and then click at the right side at Storage settings:

 

Then we click at Configurations and at the right side at Create New:

 

We select Amazon s3 and we add the following info:

Save it.

Step 5: Set the new storage at storage settings

Navigate to Storage settings and select from the drop down menu the new Amazon s3 bucket.

 

Keep only at your server your theme resources:

*Attachments and Downloads doesn't work out of the box as it seems there is some hardcoded code at Invision related to S3...

To make them work you just need to install the addon bellow: (tested with Invision 4.4.4).

 

Then wait for any Invision background tasks to finish and check.

I hope that you will benefit from this tutorial and get better performance while saving a lot of money 🙂

Enjoy!

OTHER NOTES:

If you are already using S3 then it is very easy to move straight to Wasabi by following the above tutorial as it is NOT needed to move back your files to server and then to Wasabi.

CREDITS

Credits to @Joel R for giving me the motivation to write this tutorial and providing screenshots from his Wasabi account and for editing the topic.

Thank you

Edited June 18, 2019 by ASTRAPI

[Joel R]

@ASTRAPI is helping me move my community over to Wasabi and Cloudflare.  

For any Invision Communities who are currently using Amazon S3, then you may want to seriously investigate the pricing for Wasabi or other S3 compliant services.  Wasabi is significantly cheaper (80% cheaper), charges nothing on exit bandwidth, and compatible with CloudFlare.  It's a drop-in replacement for S3, so for purposes of your ACP, you would treat it like an Amazon S3 storage.  

[Fierce God]

WoW, thank you for this info

 

What if we are on CiC or is this only good use for "Self Hosted"?

Noob Question: - 

We really want to start using our Gallery and Files apps, but know that these are the only two things that will take up disk space

Is there a way that CiC users can safely use wasabi just for our Gallery and Files app? and still be compatible with the other CiC storage they already use by default?

Hope i am asking that right? lol

Edited May 25, 2019 by Fierce God

[ASTRAPI]

Quote

What if we are on CiC or is this only good use for "Self Hosted"?

You can use it on any environment that supports natively or by any plugin/addon Amazon S3. For CiC (community in cloud) yes it will work!

Quote

We really want to start using our Gallery and Files apps

You can select from your admin CP which files you like to move and which not 🙂

Edited May 25, 2019 by ASTRAPI

[Fierce God]

2 minutes ago, ASTRAPI said:

For CiC (community in cloud) yes it will work!

 

Ok thank you, we will seriously look into this

[Steph40]

I think I followed the tutorial but it is not working, I tried to moved downloads files (got only one) and when the transfer is done and I try to download the file I get : 

This site can’t be reached ERR_INVALID_RESPONSE

I checked wasabi and the file is there. 

Maybe I misconfigured the cloudflare part? I am always confused with dns settings.

So created cname

Name : cdn

Domain name : mysite.ca.s3.wasabisys.com

 

 

[ASTRAPI]

Assuming from your settings that you are using the European datacenter:

Please double check all settings from the tutorial and most important the DNS part:

cname -> cdn -> cdn.yourdomain.com.s3.eu-central-1.wasabisys.com  <--For Europe datacenter

or

cname -> cdn -> cdn.yourdomain.com.s3.us-west-1.wasabisys.com  <--For US-WEST datacenter

or

cname -> cdn -> cdn.yourdomain.com.s3.wasabisys.com  <--For US-EAST datacenter

 

So your record should be: 

cname -> cdn -> cdn.mysite.ca.s3.wasabisys.com

 

Then give it a minute and try again....

 

I just edited my topic to clarify that entry ....

Edited May 25, 2019 by ASTRAPI

[Joel R]

1 hour ago, Steph40 said:

I think I followed the tutorial but it is not working, I tried to moved downloads files (got only one) and when the transfer is done and I try to download the file I get : 

This site can’t be reached ERR_INVALID_RESPONSE

I checked wasabi and the file is there. 

Maybe I misconfigured the cloudflare part? I am always confused with dns settings.

So created cname

Name : cdn

Domain name : mysite.ca.s3.wasabisys.com

Hey @Steph40 you may need to move off the free plan and setup a credit card on file. 

I was trying to check some of my files that were moved over, and I got a message in the console saying there isn't public access on the free plan.  You'll need to add a credit card on file.  As soon as I added the credit card, the images appeared.  No money no honey.

As another note, if you are actually performing a file storage migration from S3 to Wasabi, you should first test with a minor setting like Profile Fields.    

Edited May 25, 2019 by Joel R

[ASTRAPI]

Yes you will need to add your card and charged 5,99$ per month for 1TB storage and unlimited bandwidth to get it work with Invision.

* I did the related edit using Bold on my topic.

* I add also an example of a Cloudflare Cname record.

*I add also a notice that is better to not move to Wasabi the theme resources... For all the rest is ok.

Edited May 26, 2019 by ASTRAPI

[ProSkill]

I am getting an SSL error 526 from cloud flare. It appears that all of my attachments have been successfully moved, but they all give the SSL error. 

[ASTRAPI]

At Cloudflare on Crypto section select Full and check if it works.

If not then check again the tutorial step by step....

There is an issue with your certificate at your server:

Error 526: Invalid SSL certificate

Error 526 happens when Cloudflare is unable to successfully validate the SSL certificate on the origin web server and the SSL setting in the Cloudflare Crypto app is set to Full SSL (Strict) for the website.  

https://support.cloudflare.com/hc/en-us/articles/200721975-Error-526-Invalid-SSL-certificate

 

Edited May 27, 2019 by ASTRAPI

[ProSkill]

7 minutes ago, ASTRAPI said:

At Cloudflare on Crypto section select Full and check if it works.

If not then check again the tutorial step by step....

There is an issue with your certificate at your server:

Error 526: Invalid SSL certificate

Error 526 happens when Cloudflare is unable to successfully validate the SSL certificate on the origin web server and the SSL setting in the Cloudflare Crypto app is set to Full SSL (Strict) for the website.  

https://support.cloudflare.com/hc/en-us/articles/200721975-Error-526-Invalid-SSL-certificate

 

It was already set to full strict. I think I messed up the custom url part, I didn't include the https in the cdn url.  I am re-running the conversion, it's basically just moving it to a new bucket in Wasabi. It appears to be moving the files correctly so hopefully this will fix it. 

[ProSkill]

So, I got this to work! It's quite simple actually.  Thank you to whoever created the tutorial.  A few gotchas: make sure to set CloudFlare to Full, if you do Full(Strict) you will get the SSL error that I described above. Also, make sure to use https:// in your CDN url. Other than that it's working great so far. 

Edited May 29, 2019 by ProSkill

[The Old Man]

Great candidate for a pinned topic!

[AlexWright]

A perfect replacement! Thank you!

[ASTRAPI]

You are welcome 🙂

I am getting a lot of positive feedback about this topic and i think (if admins agree) it will be helpfull to make it a pinned topic ....?

[AlexWright]

Well drat. It was a perfect replacement (thankfully I hadn't started moving files over, was waiting to get Cloudflare setup), until I noticed in their ToS that they don't allow:

"(A) any information that is unlawful, harmful, threatening, abusive, harassing, tortuous, defamatory, vulgar, obscene, libelous or racially or ethnically objectionable, or promotes such activity"

I'm waiting for a reply from them to see what exactly is meant by vulgar and/or obscene, since we do have what could be considered adult illustrations. This does make me wonder now if there are other decent s3 replacements?

[ASTRAPI]

Anything legal should be fine 🙂

If you are not sure about your content then ask them and see ....

 

[ASTRAPI]

Just edit my topic with a note about that Invision is using something specific (header?) to S3 and only Downloads can't work with Wasabi.

Any related info? It will be great if that can be adjusted and work on both so users can benefit on both cases...

Maybe @bfarber ?

Checking also this one:

 

Edited June 13, 2019 by ASTRAPI

[Joy Rex]

If we don't use CloudFront/Flare, what would be the setup process - add a CNAME to our server per your instructions above?

[Joel R]

15 minutes ago, Joy Rex said:

If we don't use CloudFront/Flare, what would be the setup process - add a CNAME to our server per your instructions above?

Are you using any cdn?  Or no cdn?

[All Astronauts]

Signed downloads for S3 files in IPS software is locked to Amazon S3 (bucket regions and the like all hard coded). This affects attachments (the direct downloading of them, not the viewing in posts) and Downloads files.

The S3 Compatible Downloads plugin hacks around this limitation. Read the file description fully before deciding if you want to use it but yes, all good in various hoods. I, and Joel now for that matter, can confirm as such.

Full S3 compatibility for various services (Wasabi, Digital Oceans Spaces or whatever, and so on) will require new storage handlers written to handle the various signatures required AND whatever bucket regions the services use/require.

[Joy Rex]

18 hours ago, Joel R said:

Are you using any cdn?  Or no cdn?

No CDN - are there any good free CDNs you would/could recommend?

[ASTRAPI]

One of the best and free that you can use is Cloudflare as it can take care your DNS and static files like images .... and it can offer also some other optimizations !

They have almost everywhere a datacenter...

https://www.cloudflare.com/network/

 

Edited June 14, 2019 by ASTRAPI

[Joel R]

2 hours ago, Joy Rex said:

No CDN - are there any good free CDNs you would/could recommend?

One of the biggest reasons for switching from AWS S3 to Wasabi is that you can use Cloudflare. 

Even if you don't have a need for third party storage like Amazon S3 or an S3 compatible storage, you should be investigating Cloudflare anyways with your server admin.