Jump to content

S3 Compatible Downloads

You are viewing a curated collection of the most significant posts in this topic.

Featured Replies

Posted

Non-Amazon S3 storage services do work with IPS 4.1x. Add your endpoint, key, secret, and sub-directory and custom url if needed, assign to some file category such as attachments or Gallery images and off you go!

However, these Non-Amazon S3 services are NOT officially supported by IPS - that's because some of the internal methods have not been altered to handle them. These methods are hard-coded for Amazon S3 exclusively.

If you are merely displaying images and the like on the front end, none of this is a problem. It becomes a problem if your end-users are going to download these files - that is when you hit the exclusively Amazon S3 code and the downloads will fail. For example, if you assign non-Amazon S3 storage to save IPS Downloads files, those files will not be downloadable. When the system attempts to make the download URL, it crafts that URL as if you were using Amazon S3, not some other S3-Compatible storage service.

The two major areas that this will impact are IPS Downloads files and Attachments (this is what the file storage is called in the Files list in the ACP - it encompasses all attachments such as those you would have anywhere the full editor is used and there is an attachment dropzone.)

This plugin intercepts the URL creation method called on Amazon S3 file storage objects, checks to see if it is an actual Amazon S3 request, and then just passes it on through as usual. If it is NOT an Amazon S3 request, it must be an S3 compatible method, in which case I provide a rudimentary URL to patch this up. It isn't anything fancy: just enough headers to push out the actual file name and trigger your browser to download it. If the attachment or file in question is an image your browser might just pop it up in a new tab, in which case just right click and download it there. This has been tested with the usual modern browsers (minus Safari), and various file-types. All testing was successful.

BTW, as an aside, external FTP storage for files is NOT COMPATIBLE with IPS Community in the Cloud. Might want to note that...

Free, support through the IPS Marketplace topic for now. Free also means lay off the instantaneous demands for support. I use this too so it's in my interest to keep this going. Support, and this plugin's existence, continues until IPS gets around to actually officially supporting Non-Amazon S3 storage.

WARNING!

The only item in the entire IPS Suite that marks files as private is IPS Downloads. This plugin will flag them as public. This means a few things:

1) If you are monetizing or otherwise require to fully control access to IPS Downloads files this is probably not the plugin for you. You will need to wait for complete 3rd Party S3 support, or at least specific mods to support obj.space, Digital Oceans Spaces, and so on. It really is just some hump work on getting the signatures correct (and it might be a V2 vs. V4 dance with varying providers as well - IPS just has a hard-coded-to-Amazon V4 method). Give it time, it will happen.

2) If you are using this plugin on an already existing bucket with Downloads files, those files will need to be flagged as public-read for them to download correctly. Again, this plugin is just getting rid of the signatures to allow public files to download correctly from 3rd-party S3 providers. 

3) IPS Downloads users on IPS Community in the Cloud may need to wait forever and ever before the plugin will work correctly (forever and ever being defined as more than handful of minutes) as CIC caching takes some time to get things working. You *will* need to be infinitely patient. Like, install and then go drink something for awhile, read a book, etc. Then test. Don't yell at me saying this is broken., It isn't. Tested and verified on both own-hosting and CIC.

Enjoy, good luck, etc...

 

 

Edited by All Astronauts

  • 3 years later...
 

Does this work for Backblaze?

I don't think Backblaze is supported...fully at least as it seems to need a secured connection to do Content Dispostion which seems to be the issue for most of these.  For me I have been looking at multiple solutions as My Downloads egress on Wasabi was way too high like 2.5TB compared to the 150GB I have stored.  The CDN Option with cloudflare simply didn't do much at 25% cached on average and in addition to that the whole thing went up in smoke when I got a 522 I could not recover from. I tried to see if Wasabi will match some additional bandwidth cost like their competitors.  No go.  They gave me till Monday to get my bandwidth down. 3 Days.

Fast forward I decided I didn't mind paying a little extra (0.01/GiB) for additional bandwidth but I need to find a provider that works.

First I tried Digital Ocean.  Nice provider but it seems they have switched over to the dreaded must have a signed account for public read.  So this one was a no go, but I might keep it as a storage backup

Next I tried Backblaze.  And like Digital Ocean same thing, trying to download with the headers gives you an error basically saying you are not allowed to do this anonymously.  It works great for pictures if that is what you want to use it for but not so much for downloads.

Next up Linode. Nice little service but I could not get the thing to work with S3Browser, which for me was just a sign this not going to be good.  S3 Browser allows you to do things like update bucket policy and other things many providers do not have a front end for so if it didn't work, it was a no go.

Finally I tried Vultr.  They are a new Object Storage provider so I figured..."Hey Why not".  So I went off transferring my June folder over to do initial testing.  I test by basically grabbing the link that IPS uses in downloads and seeing if it will work or error out.  To my delight and surprise, It WORKED!!  This tells me vultr is pretty much setup exactly like Wasabi, and with S3 Browser any bucket policies I had I can add them directly through an API call.  Also in addition Vultr is fast....like stupid fast compared to the rest.  Backblaze I found to be the slowest of all.

So for pricing Vultr is $5/Month giving you 250GB  of storage (additional 0.02/GiB) and 1TB of Bandwidth (Additional being 0.01/GiB).  For me this turned out to be roughly $25 Per Month for the month Bandwidth I am using.  That price is not bad for the peace of mind.

Anyway wanted to put this up here as I have not really seen much convo on S3 Compatible Storage that actually works outside of Wasabi.

  • 4 weeks later...
  • Author

Well, see, the thing is, you step away from something long enough, and come back to it with fresh eyes and forgotten knowledge and stuff jumps out at ya.

Cryptic? I think I solved Wasabi finally. DigitalOcean Spaces probably as well. Both with the signed temporary urls for attachments and Downloads.

I might charge a touch as I'm probably throwing this into an app and adding in ACP widgets for the service status stuff.

Gonna simultaneously poke things and mull things over, if it is possible to do both at once.

It was such a benign, little, utterly ignorable thing too...

  • 7 months later...
 

I might charge a touch as I'm probably throwing this into an app and adding in ACP widgets for the service status stuff.

 

Was this ever released?

Recently Browsing 0

  • No registered users viewing this page.