Shortfalls in the Hooks/Apps systems

[Michael]

When IP.Board 3.0 came out, I worked hard on making sure I could convert as many of my old mods to the new setup, and seeing if they could all be done with zero file/skin edits. I was largely successful, but even at the point we're at now with the 3.1 series getting ready to be replaced, there are several old mods of mine I still have no means of upgrading. After reading through the blog entries of the new updates for the Hooks and Apps system and seeing not a whole lot I think the resource community has been asking for or can make use of, I wanted to give you a rundown on the big issues that are roadblocking the mod upgrades I still have not been able to accomplish.

Admin CP Hooks
We need to have some way to write hooks for the Admin side. We can make Admin-only applications, but there isn't any way to hook into the Admin section. Things like template hook, skin overloaders, data hooks, etc., would be a means to allow new development to take place there.

Mods of mine I can't upgrade because of this limitation:

• Ban Filter Reasons
• Reorder Emoticons

Intercepting the member creation/updating process
Sometimes it would be nice if we could hook into the IPSMember functions to do things when members are created/saved. We might want to ensure certain things are set the way we like when a new member is created on our site. Just some way to get access to those fields before there are inserted/updated would be nice.

Mods of mine I can't upgrade because of this limitation:

• Default Signature
• Require Birthdate at Registration

Set our own parser flags
There are a ton of places where the bbcode parser is used, and the various flags being set on whether things like bbcode, smilies, html, etc., can be used is hard-coded. It would be very nice if we could define for certain instances of the parser whether we wanted to allow things to parse or not. Of course, setting a default value on these is fine, but it would be handy if we could overload those when we wanted.

Mods of mine I can't upgrade because of this limitation:

• HTML Signature Permissions

Robust custom installer for hooks
You gave us the option to run a custom installer script for hooks, and that is very cool. But it is also very limited in what we can do with it. It basically just lets us run a bit of PHP code that has to execute in a single page load. That is fine for small things, but what if we have a hook that needs to update something for every member of a 10,000 member forum? I'd like to see a way we could have a custom installer for our hooks that allowed us to perform loops of actions over multiple page loads.

Mods of mine I can't upgrade because of this limitation:

• Number of Topics Started

Thanks for taking the time to look at these requests. I encourage other mod authors to add their thoughts to this. Of course, none of this will make it into IP.Board 3.2 at this point, but it would be nice if IPS would consider these things for a future version.

[bfarber]

Regarding the admin CP hooks, you can already (in 3.1.x) use action overloaders for action files. I can't speak for skin files in 3.1.x, but at least as of 3.2 you can now use library hooks to overload skin files as well.

I agree about intercepting the member creation/update process. I think that would be useful for many addons.

Overriding parser flags I think would be a relatively tricky thing IMO. I can't really see an overly clean way to allow you to override any parser flag anywhere. I will say that the editor is now loaded using a library hook, so you can probably just hook into the editor perhaps to try to capture these things. For your specific instance, it may be easier to overload a file used on every page load (i.e. skin_global) and then manipulate the setting ipsRegistry::$settings['sig_allow_html'] based on the current viewing user's group. This would ensure anywhere that the signature is parsed it would use your custom value, without requiring you to capture every specific instance of it being parsed. Just a recommendation for the specific instance you mentioned (per-group HTML signatures).

Custom installer for hooks would be nice, but potentially overkill. I haven't really seen many other requests for something like this, so I'd be curious how much it's needed before investing a lot of time into it. You could always include a standalone script that a user would need to run if it became necessary, at least as a stop-gap.

[Michael]

Custom installer for hooks would be nice, but potentially overkill. I haven't really seen many other requests for something like this, so I'd be curious how much it's needed before investing a lot of time into it. You could always include a standalone script that a user would need to run if it became necessary, at least as a stop-gap.

I've avoided doing this because it is something outside of the scope of standard hook installation, and I wanted to be completely compliant with these best practices. I may break down and do this, though.

[AtariAge]

I just had a relatively simple modification for the ACP rejected because it contained a code edit (a simple, one block, benign edit to an ACP skin file). I think it's pretty piss poor that Invision rejects modifications that include code edits when there is simply no other way to perform the modification in the current system. Invision should be doing everything in its power to ENCOURAGE people to create modifications, rather than throwing road block ups such as this.

I understand the motivation in wanting mod authors to use Invision's hooks system but to tie the hands of authors like Michael where he can't even update many of his modifications for 3.1.x and release them in the IPS Marketplace is really a shame. I don't understand why Invision had to go "All or Nothing" with this policy. Sure, if a modification can be written using the hooks system, don't approve it if it includes file edits. But if there are tasks that can't be accomplished in this framework, is the right thing to do to simply deny the author exposure in the IPS Marketplace? In my opinion, NO, especially when there are still quite a few mods in the IPS Marketplace that were grandfathered in (and continue to be updated!) that require one or more file edits (some fairly involved!)

..Al

[AndyF]

To clarify :)

My reasons for suggesting it would be better as a community article:

1. The download was in effect, just a readme file ie: there were no files supplied / required, this alone would disqualify it now.

2. It contained source code edits.


I do think it can have a home as it is in the articles section though, as indeed I suggested this in my reply. :) , I did not leave you 'out in the cold' with it, for want of a better phrase. Indeed, most of my small tweaks live in this section as they have file edits as there is no current alternative and / or they are only needed by a small section of the community who understand the risks with core file edits.

[AtariAge]

To clarify :)

My reasons for suggesting it would be better as a community article:

1. The download was in effect, just a readme file ie: there were no files supplied / required, this alone would disqualify it now.

2. It contained source code edits.

This is what most modifications were before the Hooks system came along. You downloaded a text file that described how to perform the modification, sometimes you had to make database changes, and sometimes there might be some additional files to upload somewhere in your board directory structure. The file I supplied was generated with Invision's documentation generator, so it had a thorough description of what the mod did and instructions on how to perform the singe file edit.

I do think it can have a home as it is in the [url="http://community.invisionpower.com/resources/articles.html"]articles[/url] section though, as indeed I suggested this in my reply. :)

I don't agree, I think mods should all be in the "IPS Marketplace", even if they require one or more file edits. Nobody is forcing anyone's hands in editing files by hand in order to install a modification. If someone's not comfortable doing so, they can delete the file and move on. I and others have been doing this for years. Yes, it's great that Hooks and Applications can largely supplant having to modify files by hand (and it certainly makes updating the forum easier), but it's not presently a catch-all for every type of mod, as evidenced by this thread.

The articles section is not an obvious place for "modifications". Someone unfamiliar with Invision's site structure is probably going to head straight to the "IPS Marketplace" and assume that all the forum modifications (available on Invision's site, at least) are in this one location. Unfortunately this is not the case, and I wasn't even aware of this until today. Even better, if you try searching the Resources (using the search system), you don't get any results. I was shown later that you can go into the Support->Documentation area and the articles will then be included in a search, but that sure as heck isn't obvious, nor does it help with the issue that some mods are located in this "articles" area.

..Al

[AndyF]

This is what most modifications were before the Hooks system came along. You downloaded a text file that described how to perform the modification, sometimes you had to make database changes, and sometimes there might be some additional files to upload somewhere in your board directory structure. The file I supplied was generated with Invision's documentation generator, so it had a thorough description of what the mod did and instructions on how to perform the singe file edit.

Yes, but they (apart from a few that have fallen though) had files to upload or something other than an instruction file. :)

If say I submitted this as a modification (which it is) just in a readme.txt file, I would expect it to be rejected as there is nothing to "upload", ie: no hook.

From what I see, your submission was a nicely written guide to gain extra functionality but it did not meet the requirements for the Marketplace as there were source edits and there was nothing to upload as such, hence me still saying its ideal for you to submit it to the articles.

I'm not arguing with you, just posting my thoughts. :)

[AtariAge]

Yes, but they (apart from a few that have fallen though) had files to upload or something other than an instruction file. :)

If say I submitted [url="http://community.invisionpower.com/resources/articles.html/_/ipgallery/increasing-the-uploader-limit-queue-r541"]this[/url] as a modification (which it is) just in a readme.txt file, I would expect it to be rejected as there is nothing to "upload", ie: no hook.

From what I see, your submission was a nicely written guide to gain extra functionality but it did not meet the requirements for the Marketplace as there were source edits and there was nothing to upload as such, hence me still saying its ideal for you to submit it to the [url="http://community.invisionpower.com/resources/articles.html"]articles[/url].

I'm not arguing with you, just posting my thoughts. :)

What I'm arguing is that a modification is a modification, regardless of whether you have to modify source files by hand, upload and install a hook, or some combination of both. The fact that there is nothing to "upload" to your Admin CP really should have no bearing on the issue, except that there's apparently an arbitrary rule in the submission process that states you need to have some files to "upload". I'm not arguing against hooks by any means, I think they're great! But we're being restricted on the types of modifications that can be posted in the IPS Marketplace by excluding modifications with source-file edits, given the hooks system is not yet complete enough in its scope to allow most types of mods.

..Al

[AndyF]

Well, I don't want to argue with you :) , so hopefully a staff member will have something to post here too.

My concern was simply that a readme.txt should not really be a download when it could be an article. I could quite easily turn a lot of articles I've put together (even just the 3.x ones, I would guess there must be 30 if not more) into downloads but I never saw them as suitable for 'downloads' even before the 'no source file edit', even going back to the 2.x days, would be a lot more than 30 odd then.

However, speaking as a non-coder really, while topics such as this exist where a modification author cannot easily achieve what they wish to without file edits then the hook system does need more. I'm hoping the improvements in 3.2.x will help assist mod authors with this.

[AtariAge]

Well, I don't want to argue with you :)

Nor I with you.

My concern was simply that a readme.txt should not really be a download when it could be an article. I could quite easily turn a lot of articles I've put together (even just the 3.x ones, I would guess there must be 30 if not more) into downloads but I never saw them as suitable for 'downloads' even before the 'no source file edit'.

However, speaking as a non-coder really, while topics such as this exist where a modification author cannot easily achieve what they wish to without file edits then the hook system does need more. I'm hoping the improvements in 3.2.x will help assist mod authors with this.

My concerns are mainly two:

1) Modifications are being split into two distinct areas, one of which (articles) is not an obvious area for modifications nor easily searchable.
2) Because the hooks system doesn't yet allow for all types of mods (and it may never achieve that goal), I don't feel it's fair to discriminate against mods that require file edits when that's the only way they can be achieved.

Because I had no idea mods were being pushed into the articles area, I surely missed out on various modifications that would be useful on my forum. Now I need to look at both the IPS Marketplace and Resources area when browsing the latest mods.

..Al

[AndyF]

Well (again as a non coder) I see instances of private functions where perhaps they should be public but I can only assume they are private for security reasons. :unsure:

[bfarber]

We do not want to accept modifications to the marketplace that require source-level modifications. I understand that not everyone will agree with this rule, but we do not intend to reverse it at this time. There was a time when every modification required source-level file edits. Our hooks and app systems are pretty robust now, and ever-growing, and this is not the case anymore. There are so many novices using the software that don't understand what a php file is, or what FTP is, and then you have our hosted customers who don't have access to the source code (hosted customers receive an encoded version of IP.Board, unless they purchase a license separately) so these types of modifications are problematic for much of our customer base. No one prevents you from sharing them externally, or sharing them as an article, but we don't wish to accept them in the marketplace.

We also have plans to expand the marketplace in the future to do cooler things than it can at present, and these types of mods simply wouldn't work.


That said, we do require feedback to let us know where improvements are needed. ACP skin files, for instance, can be overloaded as of 3.2 (at least - I was not aware they couldn't in 3.1, but that may very well be the case).

[AtariAge]

That said, we do require feedback to let us know where improvements are needed. ACP skin files, for instance, can be overloaded as of 3.2 (at least - I was not aware they couldn't in 3.1, but that may very well be the case).

That's good, I hope the hook system continues to be expanded and evolved over time. I'll definitely take advantage of the ACP skin file overloading once that's available in 3.2.

..Al

[AtariAge]

I just found another instance where I cannot use an Action Overloader because the "_autoValidate()" function in /admin/applications/core/modules_public/global/register.php is private. I'm not sure there would be another way to properly write a hook to do what I want here, which is to send out an email when a user email validates their account, so I know when I need to do the admin validation. I have code edits in this function now and was hoping to turn it into a hook. Taking a look at that particular class, all but three functions in it are private.

..Al

[Michael]

That should be resolved in 3.2:

Almost all private methods have been switched to protected to allow more robust hook capabilities

Source:

[AtariAge]

That should be resolved in 3.2:

Source:

That's good news, I didn't get a chance to read that until now.. :)

..Al

[teraßyte]

I can confirm that the function _autoValidate has been changed to protected for 3.2 :)


Also as Brandon already mentioned you can overload the skin acp files in 3.2, I added the hook points myself about 2-3 weeks ago.

[AtariAge]

[quote name='tera

[Michael]

Found another potentially big shortfall today. There is no single central bit of code used to generate the data that will eventually go into the userInfoPane skin template. You can construct a hook that will hook into the 'topicViewQuery' data hook location in order to ensure certain additional fields are pulled which will then be used to add data to the userInfoPane, but that template is used in places other than just the topics; it also gets used places like Calendar events and Announcements.

There needs to be a single, re-used function that grabs data that will eventually go into the userInfoPane template, and it needs to be hookable.

[bfarber]

Unfortunately, I don't think that is realistic from a resource POV. We join the member fields on to queries and then output this data to a template (which in turn passes the data to the userInfoPane template). If we had some central place that generated this template bit, it would require querying the data separately, which is undesirable because it would increase resource overhead.

You could of course work around this in the userInfoPane skin overloader hook by capturing the member ID passed in and querying the data yourself, though it would mean one query per instance of the template.

[Lewis P]

You could of course work around this in the userInfoPane skin overloader hook by capturing the member ID passed in and querying the data yourself, though it would mean one query per instance of the template.

Doesn't that also mean it's bad from a resource point of view?

As stupid as it sounds... maybe a hook point in *all* queries that relate to the userInfoPane so they can all be overloaded at once? I don't know if this is possible.. I don't use overloaders.

[bfarber]

Doesn't that also mean it's bad from a resource point of view?

As stupid as it sounds... maybe a hook point in *all* queries that relate to the userInfoPane so they can all be overloaded at once? I don't know if this is possible.. I don't use overloaders.

Yes, it would certainly not be idea for resource reasons, but we can't ship "out of the box" in a bad performance state to make it easier to modify the software. This is a fine line we walk, trying to make the software extensible while still trying to ensure that for the vast majority of users who do not use hooks or modifications that it runs as efficiently as possible. I think in this case the end result would be worse performance for the core package if we made the change as suggested.

Now, it's possible to add data hook points if specific queries are identified and brought to our attention (we would pass the query data to the hook point as an argument so that you could manipulate the query and pass it back). This would, however, require someone to compile a list of every single query this needs to be done for.