Amazon S3 / CloudFront as a scalable storage with IPB

[Nadi]

S3 / CloudFront as an Infinite (scalable) IPB upload storage device

Hi guys,

After setting up my IPB forum, I thought that the last change I should make is to make all file uploads (mostly images), to be uploaded to Amazon S3 / Cloudfront.
Amazon S3 is a scalable storage. It is great for storing large files without worrying about capacity. I can imagine that if the images capacity exceeds the EBS storage limit (up to 1 TB, which can be filled fast with high res images), I will have a problem. I though that with S3, I won't have to worry about making configuration to support multiple EBS storage devices. As far as I know, IPB doesn't support multiple storage devices for uploading files (which would be very useful for some).

From what I've read, there is a solution: Installing FUSE + s3fs and sshfs on Ubuntu, which turns Amazon S3 to look like a drive on Unix base systems (more information here). One problem with S3 is that you can only mount one S3 to an instance. This is where Cloudfront comes to an aid. Using Amazon Cloudfront you can setup 10 different CNAME for different upload locations (for example: images.mydomain). Furthermore, Amazon CloudFront caches the content and acts as CDN for first content delivery.

What I want to do is to find the best solution so integrate an infinite storage and durable storage drive to be used as a local drive for different IPB applications. The link that I posted above seems like a good start to mount a S3 storage service and map it as a drive for Ubunto OS. Again, the alternative is to use another EBS storage device as a drive for uploading user files from within the forum, IPB galleries or other applications.

If you have any experience with this, I would appreciate it a lot. I want to make changes now, and not getting into complicated migrations later. S3 will even cost less than EBS because I only pay for the storage I use, while in EBS you pay for the entire block storage space (Even if you don't use it) and S3 is a great scalable solution for storing static files like images.

Here is a comparison between S3 and CloudFront

Thanks. :rolleyes:

[Nadi]

Guys, can anyone provide any help on this?

[RogEnk]

I had a play with this, driven largely by limited disk space on a VPS.

Used the fuse/S3 method symlinking the downloads directory to my S3 mount point.

Worked quite well, downside was the double take on bandwidth - accounting for S3 > Mount > user requesting file, so my quota was eaten into by download and uploads.

What is really needed is a way to directly upload to S3 and serve from it in a more 'CDN' way.
But if bandwidth is not an issue fuse/S3 is workable.

[Nadi]

I read this article and it tells another story.