Secure File System Storage

[Xenboy]

In a number of popular web scripts like Drupal, the software has the ability to specify a non-public location for the software to store and serve files from. IPS doesn't allow this, you can only specify file system locations that are within the web root of the account (meaning that there is direct access to the files via a URL). For security, best practice is to store data outside of your web root location. This stops people from uploading files and then accessing them via direct URL and potentially compromising the account or even the server (particularly on servers where the web server does not run PHP scripts as the local user). 

It would be nice if IPS supported local file system access outside of web root. Right now, IPS also allows you to specify a local file system storage location outside of the web root, but if you use that file storage location, those items will no longer be accessible by IPS once the system moves them. If nothing else, IPS should warn people in the ACP that if you specify a file system location outside of the web root, the items won't be accessible.

[Numbered]

removed

Edited June 26, 2018 by Upgradeovec

[bfarber]

You can store files in the database which means they will no longer be directly web-accessible.

[Xenboy]

And also take a large hit in performance, plus, storing multi-GB files in a database isn't recommended, but is something the file system can handle.

[bfarber]

You did not specify these were multiple-GB files. I also would not store those in the database, indeed.

You can store files outside of the web root directory if you write a PHP script as a gateway. Look at /applications/downloads/interface/legacy/screenshot.php and /applications/gallery/interface/legacy/image.php. These are present because 3.x allowed you to store files outside of the web root directory, and upon upgrade we do not move the files to a new location (so a gateway script is necessary to read and print those files out). You just have to enter a custom URL into the file storage handler "http://mysite.com/applications/downloads/interface/legacy/screenshot.php?path=" and then the actual path of the file gets appended to the URL (at which point it is passed as a query string parameter to screenshot.php which does the rest).

In short, this is largely already possible, although I can't say if it's something we want to push to the forefront and make a common feature.

[Ibai]

On 6/27/2018 at 4:05 PM, bfarber said:

You did not specify these were multiple-GB files. I also would not store those in the database, indeed.

You can store files outside of the web root directory if you write a PHP script as a gateway. Look at /applications/downloads/interface/legacy/screenshot.php and /applications/gallery/interface/legacy/image.php. These are present because 3.x allowed you to store files outside of the web root directory, and upon upgrade we do not move the files to a new location (so a gateway script is necessary to read and print those files out). You just have to enter a custom URL into the file storage handler "http://mysite.com/applications/downloads/interface/legacy/screenshot.php?path=" and then the actual path of the file gets appended to the URL (at which point it is passed as a query string parameter to screenshot.php which does the rest).

In short, this is largely already possible, although I can't say if it's something we want to push to the forefront and make a common feature.

Sorry for the bump of a such an old topic @bfarber

Is there a way of avoid using this legacy screenshot loading? I store all my files in my server accessible from outside. Both latest files and most downloaded files are loading their images with this legacy system.

The bad point of this is that the browser cannot cache the images.

This is not happening with article's images, for instance.

Cheers,

Ibai

[Charles]

9 minutes ago, Ibai said:

Sorry for the bump of a such an old topic @bfarber

Is there a way of avoid using this legacy screenshot loading? I store all my files in my server accessible from outside. Both latest files and most downloaded files are loading their images with this legacy system.

The bad point of this is that the browser cannot cache the images.

This is not happening with article's images, for instance.

Cheers,

Ibai

I would suggest using AWS S3 storage in that case. It's designed for storage and very CDN friendly.

[Ibai]

1 minute ago, Charles said:

I would suggest using AWS S3 storage in that case. It's designed for storage and very CDN friendly.

Hey Charls, thanks for your reply.

I don't get the point of storing files in s3 in this case. I don't have any issue storing them in my server just like I do with articles attachments and images.

The thing is that when I use those File's widgets, instead of pointing to the resource directly, the core->global->thumbImage template bit is converting the URL using the legacy screenshot link.

The issue is in this line:

{{$image = ( $image instanceof \IPS\File ) ? (string) $image->url : $image;}}

Which I am able to comment. Then, the widget uses the real and direct link to the resource. However, those links returns error 500.

Any ideas?

[bfarber]

This means your file storage handler is configured to use the legacy loader (which was necessary when moving to 4.x from 3.x if you were storing the screenshots outside of your web root directory).

If those files are now web accessible, you can adjust the file storage handler custom URL option approriately.

[Ibai]

17 hours ago, bfarber said:

This means your file storage handler is configured to use the legacy loader (which was necessary when moving to 4.x from 3.x if you were storing the screenshots outside of your web root directory).

If those files are now web accessible, you can adjust the file storage handler custom URL option approriately.

Hey bfarber!

I think it might be a bug. I saw that in the past I had stored the files under /uploads and then I moved to /screenshots. When I moved back to uploads, the files were moved back to the uploads folder and the URLs came back to the static resource. However, it's not working with the widget "top downloaded files".

Anyhow, you see... only setting up the screenshots folder to the original one "solved" partially the issue.

Should I place a ticket? Or is this something I missed in settings? Where to I change the storage handler you mention?

[bfarber]

It's far easier for us to advise if we can get in and see your configuration. That said, the legacy handler is never "automatically" used. It was set during a 3.x upgrade in the file storage handler configuration, and if the /screenshots folder (in your case) is web accessible, it should not be necessary. You should be able to edit the storage handler and adjust the "Custom URL" option to remove the PHP loader.

[Ibai]

4 hours ago, bfarber said:

It's far easier for us to advise if we can get in and see your configuration. That said, the legacy handler is never "automatically" used. It was set during a 3.x upgrade in the file storage handler configuration, and if the /screenshots folder (in your case) is web accessible, it should not be necessary. You should be able to edit the storage handler and adjust the "Custom URL" option to remove the PHP loader.

Fine! That's the case, web accesible yeah.

I've searching in the ACP and I don't find any setting for that custom url. Could you tell me where is it exactly? I've looked into Advanced Configuration, under Data Storage tab

[CoffeeCake]

3 hours ago, Ibai said:

Could you tell me where is it exactly?

ACP > Overview > Files > Storage Settings (Button at the top) > Configurations

From there, look for the file system storage configuration and you'll see a toggle with a custom URL.

 

[Ibai]

Hey @CoffeeCake!

Understood!

I have 4 different locations. All of them have the Use a Custom URL switched off. It says this is for CDN so I understand it should be OFF. Am I wrong? Should I switch this on?

Cheers!

[bfarber]

As noted above, it's probably best if you submit a ticket so we can take a look directly.