FTP storage options silently removed.

[SJ77]

I guess I shouldn't say "Silently" I really don't know if it was announced or not. I certainly didn't know.

HOWEVER,

I have had FTP storage options in my back pocket as something I know I may need. Now I am finally looking at this as my golden solution only to find it's not there. WHAT???

I feel upset about this. Come on IPS!!!

Look at the before and after screen shots. Where is the FTP option?

 

 

[Morgin]

You shouldn’t even have an ftp server installed on a modern server. It is a major security hole. There should not be any need for ftp in 2018/2019. IPS should make the Amazon S3 option compatible with all s3-compatible storage options (if it isn’t already) to ensure anyone who doesn’t want to specifically use Amazon doesn’t have to and not rely on a community plugin, but they would be doing more of a disservice to continue supporting ftp storage. 

[Tripp★]

14 hours ago, Morgin said:

You shouldn’t even have an ftp server installed on a modern server. It is a major security hole. There should not be any need for ftp in 2018/2019.

Explain this please? Or at least cite your sources because this is the first I've ever heard of this. 😕 

I agree that using an unsecured FTP is bad, like where the authentication and transfers are sent via plain text but what about SFTP, the kind that requires a public key to access. You know, FTP over TLS that refuses to accept anything else? Is that a "Major Security Hole" too?

And what would you recommend instead of FTP/SFTP, instead?

[SJ77]

I know that in my case the option being removed is causing me all sorts of head aches

I’m now looking into mounting an ftp drive locally

[Tripp★]

1 minute ago, SJ77 said:

I know that in my case the option being removed is causing me all sorts of head aches

I can see why, and I think I'd be inclined to agree with you completely.

[Morgin]

2 hours ago, Tripp★ said:

Explain this please? Or at least cite your sources because this is the first I've ever heard of this. 😕 

I agree that using an unsecured FTP is bad, like where the authentication and transfers are sent via plain text but what about SFTP, the kind that requires a public key to access. You know, FTP over TLS that refuses to accept anything else? Is that a "Major Security Hole" too?

And what would you recommend instead of FTP/SFTP, instead?

Edit: I wasn’t aware IPS had SFTP support. Looking at screens now, I see it was there. No idea!

[Aiwa]

If your community is big enough to need this option, you likely have your own server or node. At which point SSHFS would be a superior option, and it encrypts data. 

Handle the file storage at the server level. 

There is also Samba and NFS. I'd only ever recommend NFS if your storage machine is only accessible via LAN. NFS is not secure. As for Samba, not a fan... Personal preference. 

[Stuart Silvester]

This topic is actually quite a good indicator of why it isn't supported (it's only supported as a legacy system for customers that upgraded from 3.x).

FTP/SFTP wouldn't be be reliable at all for the kind of files you use on your community. If you were using this I would expect that you would be reporting issues that couldn't be resolved without switching to a native storage solution.

A NFS/SSHFS approach would be far more reliable and quicker.

[TDBF]

On 12/30/2018 at 3:54 AM, Morgin said:

You shouldn’t even have an ftp server installed on a modern server. It is a major security hole. There should not be any need for ftp in 2018/2019. IPS should make the Amazon S3 option compatible with all s3-compatible storage options (if it isn’t already) to ensure anyone who doesn’t want to specifically use Amazon doesn’t have to and not rely on a community plugin, but they would be doing more of a disservice to continue supporting ftp storage. 

And why should they? We don't use Amazon or any other other cloud storage service and you would effectively be forcing us into something we don't want to use.

[AndyF]

Question (serious)

Is the fact that some still wish to use this option the concern

*or*

Is the concern that it was dropped from the product without something mentioned about it.

 

I should point out that I've not searched to see if it was actually noted or not.

 

My 2c worth of Feedback on this: To be fair to IPS they have taken huge strides with the release notes and I strongly applaud this (  ) as its very good now in that there's a lot of detail on what was changed or updated between each version, however anything such as an option that is "removed" should really be mentioned too I think.

Having said that: I do not think there is a need to explain in the release notes as to why xyz was no longer there, just a very brief line merely saying it was removed would suffice. Those who wanted or needed to find out specifics on it could post a topic or ask in a ticket etc etc.

[Ryan Ashbrook]

51 minutes ago, AndyF said:

My 2c worth of Feedback on this: To be fair to IPS they have taken huge strides with the release notes and I strongly applaud this (  ) as its very good now in that there's a lot of detail on what was changed or updated between each version, however anything such as an option that is "removed" should really be mentioned too I think.

With our new release notes policy, anything removed is also included, no matter how minor it is. From the 4.3.5 release notes:

Quote

Removed a code comment which was reported to trigger a particular hosting provider's malware scanner.

 

[AndyF]

11 minutes ago, Ryan Ashbrook said:

With our new release notes policy, anything removed is also included, no matter how minor it is. From the 4.3.5 release notes:

👍 :)

[Makoto]

1 hour ago, Ryan Ashbrook said:

With our new release notes policy, anything removed is also included, no matter how minor it is. From the 4.3.5 release notes:

 

Now I'm curious, what was the comment? lol

[Martin A.]

2 minutes ago, Makoto said:

Now I'm curious, what was the comment? lol

 

The FTP storage option was removed in 4.3.0 btw. Not mentioned anywhere in the release notes/release blog.

[Mark]

It's actually still there so people who were using it wouldn't suddenly end up with things broken. You could add a row to the table in the database where the configurations are stored if you really wanted to.... but I wouldn't recommend it.

It was notorious for causing errors where the FTP server's flood protection or other limitations would suddenly block the connection and then suddenly the community would be unable to upload anything and have other issues caused by the communication not working.

While some who knew what they were doing were able to configure the FTP server in a way that these issues wouldn't happen, it was used by such a small number of communities (like... less than 0.1%) and the percentage of those it caused irreparable issues to was so high, it just made sense to deprecate it. Especially in today's world where more robust solutions like Amazon S3 are available. Or, as you mention, a virtual drive on the webserver.

[All Astronauts]

Can we get some real clarity here? Is the intention that the IPS FTP routines are going to go away (5 series?) or are they going to remain available? There is plenty of 3rd-party stuff floating around out there (private mods) that use this stuff and knowing sooner rather than later would be preferable so we can start falling back now on the usual PHP code and route around the IPS methods.

EDIT: I know this is all about the file storage option but still, just wanting to be sure these methods ain't getting ditched (you are using them for updating so prob not)

[Mark]

39 minutes ago, All Astronauts said:

Can we get some real clarity here? Is the intention that the IPS FTP routines are going to go away (5 series?) or are they going to remain available? There is plenty of 3rd-party stuff floating around out there (private mods) that use this stuff and knowing sooner rather than later would be preferable so we can start falling back now on the usual PHP code and route around the IPS methods.

EDIT: I know this is all about the file storage option but still, just wanting to be sure these methods ain't getting ditched (you are using them for updating so prob not)

We have no plans to remove the \IPS\Ftp classes, which are used by the upgrader. But the ability to set up the system to store uploaded files like attachments on an external FTP server was deprecated in 4.3.0 (i.e. 8 months ago).

[Morgin]

9 hours ago, TDBF said:

And why should they? We don't use Amazon or any other other cloud storage service and you would effectively be forcing us into something we don't want to use.

Because s3 is a (quasi)-standardized object storage method that you can use a third party provider for, or as noted, roll your own with something like minio. 

Amazon is the market leader in this which is why the use is “s3-compatible”, but you wouldn’t need to be limited to just amazon. Per @All Astronauts plugin, there are some minor issues in the IPS s3 integration that doesn’t allow native s3-compatible without his plugin, but it seems like a fairly easy fix. 

For some people without dedicated servers, self hosted s3-compatible object storage, or using their VPS provider’s s3 compatible object storage offering, is a better option than amazon. 

Just increases the options for object storage which benefits everyone. 

After reading about how file system storage is done over ftp/sftp and @Mark‘s comments,  the conversation about ftp/sftp security is irrelevant (although you still shouldn’t run an ftp server in 2019 as it’s ancient outdated and compromised technology). My suggestion was really to increase the available options for object storage beyond hard coding to amazon. This would benefit people who used to rely on ftp and didn’t have a native virtual drive for storage. 

[AndyF]

I think the real 'concern' is not really the fact it is not there, it is the fact that it was not mentioned yes ? :)

^ The above is meant as a positive thing, please do not misread it as negative. See my 'feedback' further up the topic here if required to prove its positive.

[bfarber]

It's no longer available for new storage handling for technical reasons (frankly, it's just not very reliable through PHP).

The fact that it wasn't mentioned won't be a concern moving forward with our newer release notes processes.

[Makoto]

22 minutes ago, bfarber said:

The fact that it wasn't mentioned won't be a concern moving forward with our newer release notes processes.

On that note, thank you for this. Detailed change logs will make things easier on everyone moving forward, for sure.