All Astronauts

@bfarber One more, than I'm really out on this.

RebuildScreenshotWatermarks() queue task

Line 91 you run the code for when there is a no_watermark original copy. The method copies that no_watermark copy to an $original variable, deletes the location and thumb files, and then if there is no $watermark (meaning this rebuild method is running when someone is removing watermarks from their screenshots), updates the downloads_files_records table row by storing the url to $original in location, creates a new thumbnail copy off of $original and stores that url in thumb, and then sets the no_watermark column to NULL.

At no time do you actually delete the no_watermark file in this situation so those files are now zombies with no url stored anywhere for a FileHandler to grab them.

Yep. I'll add it as an option.

Thanks for the update. Would prefer we actually end up with a solution that provides a certain always-local (not off-server S3 or whatever) source to load "stuff". 

My reason is CORS madness and though that's not something that is necessarily IPS's problem to manage on end-user sites, it is a thing we'll hit if we lose the Interface directory as we will go from always loading and working as our things are loading from the local server to maybe/maybe not working depending on the FileStorage location being local or not and now we are getting yelled at and having to deal with all that stuff.

Others may have different reasons.

That response appears to be saying (by not saying...) that CIC users and the interface folder are not gonna fly?

Can we get a real answer on this? If the interface folder contents for apps is not going to install for CIC users a lot of us have a lot of work to do.

If this is just a hitch with Community Map then carry on but otherwise we are all going to need clarity here.

Addendum:  isValidFile() also doesn't account for record_no_watermark files, so if Downloads or the system ever hit that with a record_no_watermark file it would return FALSE of course.

fixUrls() as well but we are probably WAY beyond the need for any changes there. Or rather, I hope no one is tooling around with 4.0 right now.

EDIT: Also, in Downloads settings, the watermark image file upload helper is using core_Theme for the file extension, which it probably shouldn't.

I'll show myself out now.

It's in the ACP Marketplace now.

4.5.3

You do not account for all screenshot image copies; specifically you do not move the original not-watermarked screenshot images when those files are watermarked. 

record_no_watermark is not accounted for during the move process

End result is record_location and record_thumb files are moved, but record_no_watermark, the only copy of the screenshot not watermarked, is left behind.

If the actual OLD file storage location is still valid, that record_no_watermark file will still be retrieved as the file and url remain valid since they were not moved. Obviously nothing ideal happening here though.

I honestly did not think this was getting approved (at all). But there it is.

The statement provided before still stands. Enjoy this while it remains valid. Not "approved-Marketplace" valid, more like "functional-valid".

And also remember the use-case this was generated for. To allow for users to download attachments (and Downloads for non-monetized Downloads instances) without the need for signatures. If your needs are greater, features and security-wise, look elsewhere... 

Just an update on this for Pages Category Images. There is zero wrong with using what you have with 4.5, knock yourself out. 

That aside, I might just submit the old version and see if it will pass even though it does add a column to an IPS table - reason being that IPS does not follow the standards for loading categories consistently in Pages, meaning the standard hook point one would use to join a table on to the categories table is a no-go and I'm not sure I can slip by this without going way upstream and doing crap no one should be doing.

Remaining before submission:

Breadcrumbs
Edit first posts
Profile page cover photos
Force the Forums Post Feed Widget to Pull Only First Posts?

Star Ratings, Restore the Moderator Checkbox on First Posts will have to wait for the next version, skipping any js adds for the initial submission.

A few things are dropped due to IPS doing them, somethings just no longer needed, etc.

Some new apps are now a part of  KS:  a minor gallery thing, small downloads tweak, some blogs love.

There is some overall new stuff in places as well, plus improvements.

Let's see what the night brings...

4.5 version submitted Friday, September 18, about 4:45pm Central time. In the ACP Marketplace when it's there.

Nothing major, a few tweaks, etc.

EDIT: Early AM Monday, September 22 it is now available.

/**
 * Save translatable language strings
 *
 * @param  string|int|\IPS\Application|\IPS\Plugin       $appOrPlugin   Application key or Plugin ID

So alright, string or int. For plugins hitting the settings method or the uninstall routines will give us the plugin_id via the request var.

\IPS\Lang::saveCustom(  \IPS\Request::i()->id, 'muh_word', $values['muh_word_setting'] );

If you do that, you're setting things up for a bad time down the road. That will set both the word_app AND word_plugin entries, together, to the plugin_id value instead of NULL, plugin_id respectively.

				if ( $appOrPlugin instanceof \IPS\Application )
				{
					$appOrPlugin = $appOrPlugin->directory;
				}
				elseif ( $appOrPlugin instanceof \IPS\Plugin )
				{
					$appOrPlugin = $appOrPlugin->_id;
				}
				
				$insert = array(
					'lang_id'		=> $langId,
					'word_app'		=> ( \is_string( $appOrPlugin ) ) ? $appOrPlugin : NULL,
					'word_plugin'	=> ( \is_numeric( $appOrPlugin ) ) ? $appOrPlugin : NULL,
					'word_key'		=> $key,
					'word_default'	=> $default,
					'word_custom'	=> $value,
					'word_js'		=> $js,
					'word_export'	=> FALSE,
					'word_is_custom'=> ( $isCustom === TRUE )
				);

If you were to throw instances at this method you are probably alright, but your use of is_string and is_numeric is problematic when passing through "numerals" as that request var, passed through settings as is, will evaluate true on both of those.

\IPS\Lang::saveCustom( (int) \IPS\Request::i()->id, 'muh_word', $values['muh_word_setting'] );

Explicitly casting to INT will save you here for those you coding custom language bits with plugins right now. If you already are in the wild with a plugin and you are using the request entry raw, your end users probably have some possibly problematic lang table entries. No idea what (anything?) can happen down the road with both app and plugin entries set.

You might want to bulletproof this a little more or require instances explicitly.

Naturally this is taking time, nothing for it, and no blame or anything, they have to spin up alt-S3 service and test and so forth. 

If you gotta have it due to DL problems shoot a PM.

EDIT: If you saw another post by me after this one, apparently, you, and I, didn't... 

Ignore him.

Button pushed at 3:08pm Central USA time.

<two weeks later...>

So, I've sat on this for a week or two. Pulled some things that might make IPS grumpy, waited around for IPS patches to appear to make sure that Spacious patch alert feature was working, and so on. @Joel R has a near-final build and he seemed pretty perky about it so it can't be all bad.

Now, as far as Marketplace approvals go, there are multiple people doing the work, and it seems to be a take one and go (or take the next on the pile and go). Some stuff of mine has taken many days to get approved. One thing today leap-frogged something utterly trivial today and was approved in something like two hours. I've not had a rejection yet. Not expecting that trend to continue...

Also, this will not be approved in two hours.

Devs currently do not get alerts when a file is approved and I'm not checking 24/7 to see. I'm hesitant to update the description as that flags the file as recently updated and people download it and nothing has changed except the description and blah.

So no new description until it's actually approved.

When you see this in the ACP Marketplace, you'll know it's good to go:

I'll have to upload a metric ****-ton of new screens when I submit so I might just do a running log of changes with those here after submission. I'll be doing the submission in the morning while watching bike riders in Europe experience a tremendous amount of pain.

Lastly, as far as I can tell there is still a bug with templates not clearing out when you enable/disable an app or plugin on the ACP side. Which is to say post-install you are good to go, but if/when you go to the app page and toggle an app off, if there are hooks on templates on the ACP side, those hooks will remain ACTIVE until you hit the Spacious refresh caches button (if available) or use the support tool to do the same. If you do this with Spacious you might get some weird looking ACP (since the CSS is disabled so included images might be stupid large) but, again, support tool or refresh caches button and all is well.

Not my bug, not my problem; t'was reported in the alpha/beta tracker. Fairly warned be ye sez I.

No, there is no re-buy required for this upgrade though the work probably demanded such. Kitchen Sink, another full re-write, will probably require one just for the fact that it is moving from plugin to app.

4.5 submitted. No real changes other than a few code tweaks. In the Marketplace when it's there. 

The usual caveats about your browser being a dick and so on. I HAVE tested again to verify its all working, back and forth messages between a Chrome user with a custom sound and a FireFox user with no custom sound but pulling a custom community default sound. The first message to the FF user didn't play the sound but the second (and onward...) did and I can only assume the diabolical innards of the whole web notifications process are still roaming the Earth freely. I can't fix things if your browser decides to not play audio due to ajax timeouts and so on.

Which is to say, the ancient Sound Board note about this only swaps around the audio files and has absolutely nothing to do with the when, where, or hows of notifications being generated or what your browser does with them still apply.

Any one interested in a giant page (paginated table) with every custom notification sound on it? I'd probably add a field for users to add a Twitter length description of the sound as well. So few people recognize the opening strains of Galaga these days...

Note: description will be updated post-acceptance. Devs can still update the description while the file is pending but people just download anyways when they see it "updated" and that's not helpful.

Well... either IPS staff was reading this and did you/others a solid or the approval gods are various and unknowable (or they all have their own to-do piles...).

I've been spooling out my additions to the new marketplace just to keep the work low; for them and me. Of the (formerly) three pending, I didn't think this one would would jump over the similarly requested update to Remove Forum Title from Forum Index - that one is literally two lines of CSS - this one has at least some meat on its bones.

Who knows? Anyhoo, this was approved super-fast. Enjoy.

Also update is for 4.5 only, will not work on 4.4.10 and lower.

Submitted. All up to the ACP Marketplace gods now.

There is a chance I'll be pushing this to an application, and that's going to mean almost certainly, a re-buy (no way to sync with the new ACP Marketplace). If I have to do that, I'll be leaving the purchase price at like $10 for a month or two to ease the pain.

Why? Standards are up considerably with marketplace code reviews and there is a nominal chance, possibly reported once before but most sites using this plugin never usually hit this, that a deleted/merged member causes problems... and I can't handle those problems with a plugin - only apps have the capability to deal correctly with these hitches (well, the way IPS wants them dealt with anyways. I can get around it easily enough but I doubt it would pass review...)

On the plus side, you'll be getting more options, ACP previews, a name-only table popup option (vs. avatar display) and I think I'll throw a super wide table view option on that as well for those that are using this more for mod purposes (that might have to wait for the version after though). Yes, old data would be retained of course (database table copied over, then the plugin removed after the app is installed automatically).

I was 85% through the re-write for 4.5 when I saw my old note about this hitch and was like... yeah...

EDIT: You know what? This plugin does work fine as-is with 4.5 so all y'all are safe, it's just not going to be in the ACP Marketplace. I'm pushing this to an app, and I'm going to roll it all in: Forums, Blogs, Gallery, calendar I guess. Downloads, Commerce and Pages will be low-priority, but Forums out of the gate and then the others.

You know that thing where you casually mention something that's relevant to the problem but you are totally oblivious to the fact that you actually didn't account for that thing you just talked about? And then someone comes along and says "why didn't you deal with that thing you just talked about here..."?

That was yesterday.

@Martin A., with the oblivious assist from myself, got a handle on this. This has been cleared by at least two live sites experiencing this problem. A new version will be submitted to the Marketplace asap.

Now, just as a heads up to everyone: we are nudging, but not rushing, towards the end of the usefulness of this derpy thing. 

Amazon S3 is forcing everyone on over to the new styled urls where the bucket name is IN FRONT of the endpoint url. So, my.bucket.s3.awesomes3service.com/.... (this is virtual hosted-style) vs. ye'olde path-style urls, e.g. awesomes3service.com/my.bucket/.... 

The thing with the newer style is you gotta have wildcard support set up for your domain with your DNS settings. You gotta deal with your CDN settings. And naturally, your SSL certificate. Got dots in your bucket names? Shouldn't do that... Dashes are better but now you have some other hitches and so on. In a perfect world you would just have a single non-dotted, non-dashed alphabetic "word" for your bucket names and things work much more smoothly.

The patch here, as it were, is to account for dots in a bucket name and if present, flip things over to http instead of the likely secure SSL https. That stops all the yelling.

The point is, as Amazon S3 standards change, and web/browser security changes, and CDN requirements and changes, and so on, we're going to hit a point where this isn't going to be a viable hack anymore.

We are not there now but in a few years? Who can tell...

Like the post above says , when it's done.  There's a lot of stuff of mine that has to be gone through. Who Viewed the Topic is getting finished now. 

KS is at 70% done.

KS is also an entire re-write; its an app now, and I might require repurchase,  and that might be required anyways if switching to an app doesn't mesh with the 4.5 ACP Marketplace upgrade process, which is actually a thing.

EDIT: To expand on that last point, apps and plugins MUST have incremented the version number from the last pre-4.5 marketplace entry. It has to be bigger than the last one so the internal ACP marketplace can deal with installing and so on. But this is an app now (totally re-writtten - did I mention that...) so I don't even know if this going to work with the old Marketplace entry at all. A good chunk of you picked this up early on when it was like a $10 dollar derpy thing, and I'm pretty sure you get forever set at whatever the renew price is when you buy (that might have changed now?). If I do end up making this an entirely "new" thing I'll keep the initial price like $20 for a month or two so rebuys aren't that big of a deal. We're currently in a pandemic delay for the new ACP pricing structure anyways (IPS is moving from 10% to 20% cut due to the new everything-reviewed policy and the new swanky ACP marketplace)

Ehh.... I'd wait for the update to actually appear for S3 compat Dl plugin. If you aren't running a CDN you are probably fine as is anyways.

I'm going to amend the above by saying I'll post an updated file here. There was a single change in the way IPS does a url bit with 4.5 that does make a difference and can/will mess things up slightly with CDNs and SSL Certs and so on. Unless you diffed the files you'd probably miss the change.

Just wanted to throw an update out to y'all about this plugin.

Is it 4.5 compatible?
Yes. Nothing has changed massively to break things - it still does the rudimentary things it does. It is still free.

Will I put this in the 4.5+ ACP Marketplace?
Maybe... 

Maybe...?
There are two hitches and they have nothing to do with this plugin. 

1) I've had a second person hit me up with certificate (SSL) problems and CDN stuff and Downloads (that's what the above post was about). Nothing has changed on the plugin side. Also, I've tested a bunch locally without a CDN and everything is fine and since all the errors people are throwing at me are cert problems and CDNs that's going to be me out sorry to say. Make sure you have the bucket name and S3 service url cleared in your certs and possibly your DNS and CDN settings. Don't know what to tell you other than that the web and security and browsers and standards of what will be allowed and not throw errors change.

BTW, if you do get a cert error or whatever downloading something from Downloads while using this plugin and you use Chrome - hit the advanced button on that error page - it will have a still want to go to this link link - click that and you'll probably get your file - again showing the hitch is not with the plugin but certs and dns and cdn permissions and etc....

2) Did you know that most Amazon S3 compatible services are, lol, NOT fully compatible? There is a reason why IPS has not added additional providers beyond Amazon. There is ALWAYS, (I'll say it a second time because it is hilariously, depressingly, true) ALWAYS something. For Wasabi, as an example, they do support chunking of file uploads, which you will notice IPS added to the S3 storage service with 4.5. You'll be pleased to know that Wasabi does it differently (chunks upload and then you 'compose' the final file). I refer you back to the first sentence of point #2.

Will I support Wasabi, or other service, chunking with this free, ancient, derpy thing?
No... It's a free, ancient, derpy thing that let me patch up Wasabi use on my end for my use case (files not that large, no CDN) that I figured others might find useful. I made one test for chunking locally with Wasabi and it errored out trying to save the first chunk or trying to deal with the second chunk (this using the internal IPS S3 methods of course) and that's all I tested, and will test for now. Beyond this I would write a formal S3 handler-per-service application but Marketplace release for these things is probably touch and go - it's a core, fundamental, aspect of the software (files) and letting 3rd party file-things loose on the community might be frowned on now - I do not know for certain (or at all) actually but there is a risk with endusers disabling or uninstalling things with files still existing in places, or the support tool letting you disable all 3rd party apps and plugins and then stuff gets wrecked and so on. The ability for a site to get itself wrecked probably trends up with these.

If it is an app there is a flag you can set on the app itself that on install will mark it as "forever on and uninstallable" that would mitigate some of this but does IPS want any part of that in the Marketplace? These things would get so much side-eye from the support team.

Did you know there once upon a time was a Azure file handler in the Marketplace? Long time ago... What happens when the 3rd party disappears? I suspect IPS might be content to just let things be and the more needful, larger, corporate, etc., sites can get what they need privately from 3rd parties and the support can stay way out of IPS's hands. And yes, there are 3rd party private S3 compatible service apps out there - shh, don't tell anyone - no I won't tell you who has them. It's different when you have a devops team and a plan and $$$ resources and did I mention having a plan and knowing what you are doing vs. one guy throwing this stuff into the marketplace and then someone building up a site gets this and yes, it's great you are building your awesome site up and want to save money but I'm sorry you disabled this on upgrading and Gallery is trying to make new copies of files on upgrade and can't read the files because you disabled the file handler and now all the urls to your files are NULL and....

So can I submit this to the 4.5 ACP marketplace - sure. Do I want to.... eh... I'm more inclined to attach it here and be done with it. 

If you want to support large file uploads and still use this, you will need to, as you've had to all these years, PHP tweaks to upload_max_filesize, post_max_size, memory limit, execution time...

Yep yep. Should be there soon for some values of soon.