Download linked images

[pxg.me nick]

I think when you paste a link to an image, the forum should download the image from the source and treated it as a file that the user uploaded. If the image source is deleted, the image will still exist in the forum. Some images will load faster, especially on forums that use a cdn. Maybe this could be added as an option.

[Nathan Explosion]

Enable the image proxy functionality, and set the cache to indefinitely.

[pxg.me nick]

11 hours ago, Nathan Explosion said:

Enable the image proxy functionality, and set the cache to indefinitely.

This still serves the image through applications/core/interface/imageproxy/imageproxy.php?img=

Instead of cdn.mywebsite.com/uploads/whatever

[Nathan Explosion]

That's because they're not uploads, they are linked images.

The suggestion solves 2 of your issues:

1) Serves locally and not remotely, going forward.

2) If the source is deleted, then it still gets served as it's locally hosted.

[pxg.me nick]

5 hours ago, Nathan Explosion said:

That's because they're not uploads, they are linked images.

The suggestion solves 2 of your issues:

1) Serves locally and not remotely, going forward.

2) If the source is deleted, then it still gets served as it's locally hosted.

I have used the ssl image proxy with indefinite caching since before it was implemented in IPB and was a plugin. I am suggesting that linked images are downloaded from the source by the forum and treated as if the user uploaded the image. I want all images on my forum to be served through my cdn.

[h2ojunkie]

I've been fighting with this exact issue myself, and it seems the obvious solution is being missed here by IPS.  Proxy images should be store as just another media tag so the URL to the image can be whatever URL is configured in files storage. This would let us simple create a S3 bucket (or using an existing bucket) and directly link to that bucket/custom URL to serve the proxy images.

You already let us choose to store the images in any of the file storage option locations we create. But, it's forced to display using the proxy script. If we have a custom storage option as the place to store proxy images (such as s3) - it would be so much nicer if we could just set it to serve the image directly from the bucket/cdn/whatever custom storage location using whatever custom URL we choose to set when setting up that storage option. 

1) as it stands now, the file already get's stored in a bucket (if that's the storage option we selected proxy images). But, to serve that image, the server must go fetch that image from the proxy and pass it through our web servers to the end user. That's essentially adding unnecessary delay for the end user to view the proxy images, and it's creating extra work for the server that isn't' needed. Yeah, I get that the script is doing other things like making sure the image actually is still in the proxy cache, and re-downloading it from the source if we don't have it in the cache - but there's a table for that. If it's in the table, then it's in the proxy cache, so why not let the URL it serves from be direct from the CDN we have those images stored on?

2) as other users have mentioned - with an S3 bucket as the storage point - we can in-effect set the proxy to unlimited and just let all the proxy images get stored in a bucket forever if we wanted. 5 years down the road when the original source image has long been deleted - we still have it and it's still serving directly from our unlimited storage bucket. S3 storage is insanely cheep. I had 32 million revisions of images stored in a bucket- the actual storage portion of the fees was under $5 a month. If it's an old topic, those images are rarely viewed - so the costs wouldn't be much to website, but the benifit of not having dead images around the site in old topics is invaluable. 

just make proxy another media tag - when we change storage options, let the URL those images serve from change based on the configured storage option. Less server resources - faster page loads for the user, and we have the option to store external images for as long as we want ensuring no topic ever ends up with broken/missing images.  99% of the work is already done with the proxy setup you guys have. All that is left is the last step of letting us control url to that image. 

[inkredible]

@Rhett told me in a ticket that they are working already on such a feature in order to avoid the mixed content problem. 

 

"On embedded items from past posts, they will show a cert error, we are working on a new feature to rebuild old items for a future version, it's not yet an option though, so past items that are remotely linked to non https images and embedded on older posts will show a cert error with 100% ssl. "

[Rhett]

30 minutes ago, inkredible said:

@Rhett told me in a ticket that they are working already on such a feature in order to avoid the mixed content problem. 

 

"On embedded items from past posts, they will show a cert error, we are working on a new feature to rebuild old items for a future version, it's not yet an option though, so past items that are remotely linked to non https images and embedded on older posts will show a cert error with 100% ssl. "

That was top secret, now I'm going to have to take you out       

[inkredible]

Uhh sorry, next time better tell me  . I could edit the tag - Whom should we play tricks on? 

[Carl Maltby]

Any news on where this is going?

[Rhett]

1 minute ago, Carl Maltby said:

Any news on where this is going?

This feature exists in 4.2, you can rebuild past content when enabling the image proxy. 

[Carl Maltby]

Thanks Rhett.

Ever since remote image caching was introduced, the idea of permanently caching remote images has been bubbling under the surface and I think that it would make sense to introduce some method of automatically or electively shifting remote images from the cache into discrete embeds. Unfortunately, I do not have the experience to do this myself for our own site as my skill set lays more in content generation than coding work, and I am sure that this lays more in the code we can't easily work on when it comes to cloud-based sites.

Obviously there are implications of intellectual property rights when moving images from being merely remotely referenced to being discretely "site-side" which I presume will be best dealt with as part of a member's signup and site terms of usage.

Does this figure into the roadmap at all, and would it in fact be useful to push into the development suggestions? I'm not really hands-on enough to know whether this is something that better lays as third-party development or within that of the suite itself.

This is a strong and frustrating concern, especially since the remote image cache settings have been themselves reset during suite updates (4.2.x being the most recent) and support staff maintenance, resulting in our otherwise extensive (and important) cache of remote images having been dumped for re-caching when most don't necessarily exist any more. The recent changes to Photobucket remote image hosting has only made this a larger priority. As a site that is hosting almost 15yrs of educational tutorials and how-to's, a reliable method of fending off image link rot is worth its weight in gold. We've already lost way too much to Internet entropy and need to ensure that our library of forum posts can be preserved!

Apologies for the large post. 

[Dennis_87]

On 3/14/2017 at 8:12 AM, h2ojunkie said:

I've been fighting with this exact issue myself, and it seems the obvious solution is being missed here by IPS.  Proxy images should be store as just another media tag so the URL to the image can be whatever URL is configured in files storage. This would let us simple create a S3 bucket (or using an existing bucket) and directly link to that bucket/custom URL to serve the proxy images.

You already let us choose to store the images in any of the file storage option locations we create. But, it's forced to display using the proxy script. If we have a custom storage option as the place to store proxy images (such as s3) - it would be so much nicer if we could just set it to serve the image directly from the bucket/cdn/whatever custom storage location using whatever custom URL we choose to set when setting up that storage option. 

1) as it stands now, the file already get's stored in a bucket (if that's the storage option we selected proxy images). But, to serve that image, the server must go fetch that image from the proxy and pass it through our web servers to the end user. That's essentially adding unnecessary delay for the end user to view the proxy images, and it's creating extra work for the server that isn't' needed. Yeah, I get that the script is doing other things like making sure the image actually is still in the proxy cache, and re-downloading it from the source if we don't have it in the cache - but there's a table for that. If it's in the table, then it's in the proxy cache, so why not let the URL it serves from be direct from the CDN we have those images stored on?

2) as other users have mentioned - with an S3 bucket as the storage point - we can in-effect set the proxy to unlimited and just let all the proxy images get stored in a bucket forever if we wanted. 5 years down the road when the original source image has long been deleted - we still have it and it's still serving directly from our unlimited storage bucket. S3 storage is insanely cheep. I had 32 million revisions of images stored in a bucket- the actual storage portion of the fees was under $5 a month. If it's an old topic, those images are rarely viewed - so the costs wouldn't be much to website, but the benifit of not having dead images around the site in old topics is invaluable. 

just make proxy another media tag - when we change storage options, let the URL those images serve from change based on the configured storage option. Less server resources - faster page loads for the user, and we have the option to store external images for as long as we want ensuring no topic ever ends up with broken/missing images.  99% of the work is already done with the proxy setup you guys have. All that is left is the last step of letting us control url to that image. 

Yes this is needed. Our site with 200k++ images saved is really taking a big hit on performance on how images are severed now by imageproxy. 
@Rhett any plans to fix this so images are loaded directly?