Delete original images - option needed

[Gauravk]

Images gallery should give provision to delete the original images with a disclaimer that rebuild images option will be disabled if you discard the original images. 

For people who have tons of gallery images and most will never be rebuilt, so its pointless in saving 10-15 MB x 100 images per week and chocking up the server disk space

[jair101]

+1

I have raised the same issue before. Last time I heard, IPS is considering to offer a separate data storage for the originals, something like Amazon Glacier. 

It is extremely ineffective to keep original images wasting space on the live and expensive storage, when they belong somewhere in the glacier dungeons. 

[Gauravk]

To have alternate OPTIONAL storage option is good as far as it won't be forced by spending additional expense. I simply want an option to delete original as soon they are uploaded and resized optimized version is saved. In today world each DSLR camera is capable of producing 10-20 MB pics and smartphone produces 4-8 MB pics, good for print but not for the web. Normal user don't understand these image resizing concept and upload 100 x 10-20 MB pics every week, not even one ever need to be print and if system saves Full HD image under 200-300 KB is all we ever need. So the user can still print in A4 size but won't be able to produce a building size billboard.

[Optic14]

I can only assume the reason for the current behaviour is to allow rebuilds for those that use Watermarks (it wouldn't be great for quality if you changed the thumbnail size / large image size and also change the watermark image later on).

[Gauravk]

5 minutes ago, Optic14 said:

I can only assume the reason for the current behaviour is to allow rebuilds for those that use Watermarks (it wouldn't be great for quality if you changed the thumbnail size / large image size and also change the watermark image later on).

I agree, so the system can save one more copy of the largest dimension set by admin without any watermark for the purpose of rebuilding in case of watermark changes.

Still better to save 200-500 KB files than 18-20 MB.

[Joel R]

This morning I went into my ACP > System > Files to look at my Attachments table for the first time in years and sorted by filesize.  

I'm sickened by some of the absurd file sizes my users are uploading.  I have one user uploading images to his status updates that nobody reads and each image is 5 MB!  ?

 

Our boards are becoming more visual than ever before.  You need images for everything ... cover photos for profiles, images for social media promotion and shares, cover photos for blogs, screenshots for download files, attachments in forum topics.  

Our users are also taking photos that are larger and larger and uploading them to our boards. 

There's a disconnect there.  Every major web service is optimizing and compressing images and here we are, saving three versions of every image!  

[opentype]

51 minutes ago, Joel R said:

Every major web service is optimizing and compressing images and here we are, saving three versions of every image!  

Not necessarily. You can keep the global image attachments sizes as low as you want with “Maximum image dimensions to save”, “Maximum image dimensions to display” and your image compression settings. 

[Gauravk]

VERY Nicely explained @Joel R I'm 100% with you on this as I have been seeing and manually deleting 14 MB images saved from DSLR directly uploaded to the gallery. This eats up the server space too quick and then daily backup become enormous size and fill up remote back up storage in no time. 

@opentype this POST and Joel R reply are not for displaying 5 or 14 MB images, of course, we know that display images are optimized, but the concern is to keep holding on to the original mega size images seems pointless and now is the time IPB need to come up with a solution for this concern.

1 hour ago, opentype said:

You can keep the global image attachments sizes as low as you want with “Maximum image dimensions to save”

 

This doesn't work the way it says in plain English - “Maximum image dimensions to save”, I have restricted 1800 x 1200 maximum size image and still, people upload 4200 px. 

English correction is really required in what it says and what it really does....!

 

[opentype]

1 minute ago, Gauravk said:

 

@opentype this POST and Joel R reply are not for displaying 5 or 14 MB images, of course, we know that display images are optimized, but the concern is to keep holding on to the original mega size images seems pointless and now is the time IPB need to come up with a solution for this concern.

Wait! Are we talking about attachments or Gallery now? The attachment settings clearly state that the “the original image will be discarded”. 

[Gauravk]

[bfarber]

You could store Gallery images on S3 - the originals are rarely ever requested so bandwidth would be minimal and general storage costs with S3 aren't much, plus you'd benefit by having the regularly accessed images served by a CDN. Unfortunately, deleting the originals entirely isn't really feasible. For instance, Gallery 4.3 used new sizes for most images which meant the images had to be rebuilt, which isn't really possible without the original.

[jair101]

4 minutes ago, bfarber said:

You could store Gallery images on S3 - the originals are rarely ever requested so bandwidth would be minimal and general storage costs with S3 aren't much, plus you'd benefit by having the regularly accessed images served by a CDN. Unfortunately, deleting the originals entirely isn't really feasible. For instance, Gallery 4.3 used new sizes for most images which meant the images had to be rebuilt, which isn't really possible without the original.

This could work, when S3 is a viable option. In my country S3 is much slower than having the images stored on a local server, that takes advantage of the local high speed peering. Neither S3 or Cloudfront have enough pops worldwide to make them the default obvious choice. Not to mention the related costs. 

[Joel R]

1 hour ago, bfarber said:

You could store Gallery images on S3 - the originals are rarely ever requested so bandwidth would be minimal and general storage costs with S3 aren't much, plus you'd benefit by having the regularly accessed images served by a CDN. Unfortunately, deleting the originals entirely isn't really feasible. For instance, Gallery 4.3 used new sizes for most images which meant the images had to be rebuilt, which isn't really possible without the original.

I can understand the concept of rebuild, but I don't see why you MUST have the originals.  

I upload an image that's 10 MB and 3000x3000 pixels with my new iPhone.  The software resizes it to the 'large dimension' of 1280x1280 and deletes the original.  

Let's say I need to rebuild, for any reason.  If the target size is larger than 1280x1280, then keep the image as is.  It can't get any larger.  If the target size is smaller then reduce as needed.  

I understand the inherent risks of deleting the original.  But, I mean, I'm not the photographer.  I'm a webmaster trying to balance my storage costs at Amazon with growing appetite for rich media.  If I can shave off a fraction of a penny of storage costs from each of my 240,000 images in Gallery, then you bet I'd be willing to be yelled at by my users about not saving high-res image while I'm rolling around in thousands of saved costs.  

[Gauravk]

52 minutes ago, Joel R said:

I can understand the concept of rebuild, but I don't see why you MUST have the originals.  

I upload an image that's 10 MB and 3000x3000 pixels with my new iPhone.  The software resizes it to the 'large dimension' of 1280x1280 and deletes the original.  

Let's say I need to rebuild, for any reason.  If the target size is larger than 1280x1280, then keep the image as is.  It can't get any larger.  If the target size is smaller then reduce as needed.  

I understand the inherent risks of deleting the original.  But, I mean, I'm not the photographer.  I'm a webmaster trying to balance my storage costs at Amazon with growing appetite for rich media.  If I can shave off a fraction of a penny of storage costs from each of my 240,000 images in Gallery, then you bet I'd be willing to be yelled at by my users about not saving high-res image while I'm rolling around in thousands of saved costs.  

That's the same explanation I'm giving support since mid of 2017, then they said in 4.X we will see what optimizing can be done. Now they are saying put in feedback, means they either they need more people votes for this feature to develop or IPB is stuck in some situation and need time for this.

[bfarber]

Have you ever heard of a "copy of a copy"?

If you take an original image and manipulate it to downsize it, storing that copy at ~80% quality, it'll be ok to view. But then if you take that image and downsize it, and store the new copy at 80% quality, you start seriously losing image quality fast.

[Joel R]

I mean ... You bring up a good point ... maybe ... 

1. I'll be swimming in a pool filled with gold bullion coins from all the savings that I'm not paying to Amazon, and I feel like a swimming pool of gold is a fair tradeoff for image degradation.  

2.  I think the real cautionary tale is the image quality reduction. If I were to set my reduction at 90%, then a new copy of 90% of a copy of 90% is 81%, which is still sharper than your 80% reduction.  Also, I'm already pre-compressing my images before upload, then using Kraken which is shaving off another 7%, then allowing IPS to reduce it, so I'm surprised that  anyone can see anything  at that point. My images should pretty much all be shapeless blobs, and reducing it by another 80% is no big deal then.

3. If the image degradation gets too bad, then I can always yell at IPS to build in a feature to save the original after yelling at you to delete the original.  But now I'll have some gold coins to heckle you with? 

4. @Gauravk already stated in his OP that rebuild would be disabled.  

On a serious note, I can understand how some admins want to save the original for the off-chance that they want to rebuild their images to a different size.  But I find it to be a tax on my website where I have to save ultra large images or ultra large attachments.  

[jair101]

It doesn't have to be either this or that. Just add the option to save the originals at another place. Or even put them in a separate directory on the server. Find a way for us to admin to identify the originals. After that If the admin wants to delete this directory from time to time - he can go ahead, if he wants to download it locally - go ahead, if he wants to upload it to dropbox/s3/ or whatever - go ahead. 

I am getting tired to hear how beneficial it is to keep these humongous files, like we do not know better. 

[bfarber]

Keep in mind we cater to many different audiences, and not all of them are technically inclined. ? The idea was raised and discussed, so don't take my responses as "no this isn't happening". I'm simply trying to explain why things work the way they do presently.

[Joel R]

So here's my ultimate gripe: I'm paying Amazon every month for storage.  Like, literally for the rest of time for as long as my website is up.  

To save even $1 per month can make a difference when you multiple that by the rest of time.  

Here are some real back-of-the-envelope calculations for my site:

In my case, I have 250,000 images in my gallery.  I have slightly less than 150,000 regular attachments. So we have:

250,000 original gallery

250,000 reduced gallery

150,000 original attachments

150,000 reduced attachments

We're going to assume all of these files are the same size, even though originals are in reality much much larger.  

If I delete just the original images of my gallery, I'm saving ~30% of my Amazon bill, which is about $40 / mo.  That's $12 /mo.  I maintain a 3 to 5 year outlook on my site, which means I'm going to save $432 to $720.  That's a substantial sum of savings for a website thats broke poor.  

From a technical and user perspective, I can see how there's a possibility of accessing the original image though.  That's why I really like the above suggestions of making a new filestorage option for originals, where we can move them to something like Amazon Glacier that's infrequently accessed.

[bfarber]

22 hours ago, Joel R said:

From a technical and user perspective, I can see how there's a possibility of accessing the original image though.  That's why I really like the above suggestions of making a new filestorage option for originals, where we can move them to something like Amazon Glacier that's infrequently accessed.

I think this is a workable solution to the concerns raised, personally. Even if you didn't use Amazon, you could plug up another cheap HDD on the server and push the "original" images there (as a cold-storage type setup).

(FWIW, we don't save originals of attachments - only the "maximum size to save" and a thumbnail, based on your ACP settings. Thus this concern is mostly isolated to Gallery which necessitates a more balanced approach when it comes to image quality and retention)

[jair101]

3 minutes ago, bfarber said:

I think this is a workable solution to the concerns raised, personally. Even if you didn't use Amazon, you could plug up another cheap HDD on the server and push the "original" images there (as a cold-storage type setup).

I also think this covers all the bases. 

[Joel R]

1 hour ago, bfarber said:

I think this is a workable solution to the concerns raised, personally. Even if you didn't use Amazon, you could plug up another cheap HDD on the server and push the "original" images there (as a cold-storage type setup).

When are original gallery images accessed?

 I think they're accessed when a user clicks on 'download', right? This means cold storage may not be appropriate because users can be accessing original images all the time unless IPS changed the download function to point to the large image and bypassed the original.  

[bfarber]

If we implement this change, then yes we would need to review our uses of the original image and potentially adapt some areas.