Speedy Search and Stream Results

[All Astronauts]

Invision Community search and stream results push out the ENTIRE CONTENT ITEM TEXT to the web page and then truncates the result with the browser via Javascript.

This leads to long page load times, snapping screen behavior when the js routines truncate the text after the page has loaded, and so on.

This plugin truncates these results BEFORE they hit your user's browsers, giving them faster load times and better engagement.

Speedy Search and Stream Results (SSSR) is a roll-up plugin that includes my free Truncate Stream Items plugin and adds on a truncate search results feature, a bit that allows you to select the number of images pushed out to stream results when the stream item is a Gallery album update (new images posted), and a setting to allow you to truncate down stream descriptions which can get ridiculously long when a user has selected many forums and so on for their custom streams. 

The IPS javascript truncate functions will still fire on these pages but with vastly less text to deal with! Plus of course, the added benefit that that text was never sent to the page in the first place... 

See the screenshot for settings.

 

 

[Joy Rex]

This sounds really useful - could you provide a before and after example?

[All Astronauts]

SSSR 1.1 Released!

• Fixed a default setting that was beyond form number limit.
• Removed setting for word count on Streams.
• Implemented a much more aggressive filter on streams.

That default setting may or may not have messed with saving the plugin settings. Sorry bout that if it did.

The new aggressive settings is courtesy of one of my sites where the content items didn't begin with 'words' (don't ask...). While looking at that hitch I quickly realized that playing with customizable word count limits and so on wasn't really necessary and just cut this down to the bone. I looked at the three line stream results (where three lines of text are displayed for the item - that's the max right now, there are some  - like Pages database records - that push out just two lines), worked up a reasonable character count for those, and now we just limit and truncate right around that amount. Stupid fast now.

@Joy Rex without me working up animated gifs this is hard to do, so I'll just try to better explain.

The bottlenecks here are: 

• The amount of text these search and stream results push out of your server and down the internet tubes.
• This large page size now being ingested by your end-user's browser, in addition to any other page load blocking items trying to spark up.
• The IPS javascript truncation routines that fire AFTER the entire page has loaded.

This plugin addresses all three by limiting all the stream and search results text that those things have to deal with. Remember: the hitch is that by default the ENTIRE content item text is pushed out here, no matter how large it may be. That means ALL of that text is rendered by the browser before any javascript truncation happens.

• Truncating this text server-side is stupid fast and easy to do. That's less for your web server to push out the door.
• That means smaller pages for your end-user's browser to ingest  - that's a faster page load time.
• Those javascript truncate routines have very little to do now. 

If the vast majority of the text content of your community, that's forum posts, image comments or descriptions, etc. routinely consist of a handful of lines of text - seriously, like 2, 3, 4, 5 lines or something, you won't see too much improvement here. People aren't so much discussing as they are chatting. 

If instead you have in-depth discussions, people posting paragraphs of text, Pages databases where content items are walls of text (book chapters, creative writing, subtitles, articles, and what not) this is the thing that you NEED! Any animated gif I provide would, with the appropriate content items of sufficient length in the streams, show a before where you literally see walls of text appear on the screen only to eventually snap away Thanos-style when the js truncation fires - after the page has fully loaded of course. The after gif would be faster page loading and minimal, if any screen-snap occurring from the js truncation.

I can even see, for streams, a way to all but eliminate any firing of the js truncation completely. I'd just need to look at the default stream templates to see if they are wanting one, two, or three lines of text - the item type is already there, I can just read it and adjust the cut as needed. I'm noting this for a future rev. No screen snaps allowed (eventually when I get on this addition)

FYI, I've had this stuff laying around for a few years now - I just got around to cleaning it up and piling it all together rather than the hodgepodge of plugins it was.

[All Astronauts]

The thing about ideas is that once they hit, then the OCD kicks in and you have to act on it...

SSSR 1.2 Released!

• Pages CMS records in Streams default to a two-line truncation. SSSR now checks on those and more-aggressively truncates to match the usual two-line character amount. The javascript truncation routine will rarely, if ever, fire on those now. Demonstrably better (at least for my sites where this matters...)

[Steph40]

LOL I like it. Thanks for helping non coders.

[Joy Rex]

21 hours ago, All Astronauts said:

SSSR 1.1 Released!

• Fixed a default setting that was beyond form number limit.
• Removed setting for word count on Streams.
• Implemented a much more aggressive filter on streams.

That default setting may or may not have messed with saving the plugin settings. Sorry bout that if it did.

The new aggressive settings is courtesy of one of my sites where the content items didn't begin with 'words' (don't ask...). While looking at that hitch I quickly realized that playing with customizable word count limits and so on wasn't really necessary and just cut this down to the bone. I looked at the three line stream results (where three lines of text are displayed for the item - that's the max right now, there are some  - like Pages database records - that push out just two lines), worked up a reasonable character count for those, and now we just limit and truncate right around that amount. Stupid fast now.

@Joy Rex without me working up animated gifs this is hard to do, so I'll just try to better explain.

The bottlenecks here are: 

• The amount of text these search and stream results push out of your server and down the internet tubes.
• This large page size now being ingested by your end-user's browser, in addition to any other page load blocking items trying to spark up.
• The IPS javascript truncation routines that fire AFTER the entire page has loaded.

This plugin addresses all three by limiting all the stream and search results text that those things have to deal with. Remember: the hitch is that by default the ENTIRE content item text is pushed out here, no matter how large it may be. That means ALL of that text is rendered by the browser before any javascript truncation happens.

• Truncating this text server-side is stupid fast and easy to do. That's less for your web server to push out the door.
• That means smaller pages for your end-user's browser to ingest  - that's a faster page load time.
• Those javascript truncate routines have very little to do now. 

If the vast majority of the text content of your community, that's forum posts, image comments or descriptions, etc. routinely consist of a handful of lines of text - seriously, like 2, 3, 4, 5 lines or something, you won't see too much improvement here. People aren't so much discussing as they are chatting. 

If instead you have in-depth discussions, people posting paragraphs of text, Pages databases where content items are walls of text (book chapters, creative writing, subtitles, articles, and what not) this is the thing that you NEED! Any animated gif I provide would, with the appropriate content items of sufficient length in the streams, show a before where you literally see walls of text appear on the screen only to eventually snap away Thanos-style when the js truncation fires - after the page has fully loaded of course. The after gif would be faster page loading and minimal, if any screen-snap occurring from the js truncation.

I can even see, for streams, a way to all but eliminate any firing of the js truncation completely. I'd just need to look at the default stream templates to see if they are wanting one, two, or three lines of text - the item type is already there, I can just read it and adjust the cut as needed. I'm noting this for a future rev. No screen snaps allowed (eventually when I get on this addition)

FYI, I've had this stuff laying around for a few years now - I just got around to cleaning it up and piling it all together rather than the hodgepodge of plugins it was.

Thanks for the explanation - sounds more like the team here at IPS should roll this into 4.5!

[All Astronauts]

SSSR 1.3 Released!

• NEW! Forgot that Profile views have that content streams stuff there. Profile view streams now truncated as well.
• Small efficiency improvement on stream results.
• Should have clean ending word breaks on stream results now.

[All Astronauts]

SSSR 1.4 Released!

• Clubs view page streams added.

[Steph40]

I can't keep up with you! Good job.

[All Astronauts]

There shouldn't be too much left honestly. I think I still need to try and get word boundaries up on search results but it isn't a big priority right at the moment. The rest of it is the usual version one 'oh yeah, I forgot about that thing and that other thing over there and I can probably change these ten lines down to five...'

@Joel R brought up the Clubs miss a few hours ago and I was like "Oh yeah... Clubs..."

[All Astronauts]

Those of you paying attention to the release notes for 4.5 will note that this was more or less knee-capped by IPS. 

On the plus side you guys are only out 6 bucks to me. On the minus side their implementation is bare-bones, annnnndddd.... I'm not entirely sure I can claw it back. I'll have to take a look deeper - I only checked to see what they were doing and not if I can hook around and patch in or what not.

Short version? They did what I did and truncate search/stream results before pushing out. However, they are now just taking the first 600 characters of the content item - period. For stream results that's a nothing-burger. Sure, my plugin lets you tweak the amount pushed before truncation but for stream results it was always the first characters of the 'thing' that you got.

Unfortunately, this is a big hitch for search results. If your search term hit in a result is beyond the first 600 character limit, the snippet in the search result will not have any search terms to highlight. If you get lucky and the search terms are in the first 600 characters you still get the highlight of course, but chances are and so forth and etc.

My plugin went out of the way to actually get the bit pushed out to at least hit on some of the search terms (simple search in the content item to match, then take x-amount before, x-amount after to get a decent amount of text, and then push it out).

There's still the stream description truncation bit and the gallery stream image count thingy - I'll check in awhile and see what's what with those - I imagine those are fine and if I can't patch the main junk I'll still clear the decks and get that stuff out for 4.5 minus the whole main thing this plugin does :(

Just giving you all a heads up. I'll take a look a little later on to see if I can get the search results page back up to snuff but it's not a priority at the moment given all the other junk I have to crawl through to get up to snuff for 4.5.

 

On 2/27/2020 at 2:00 PM, Joy Rex said:

Thanks for the explanation - sounds more like the team here at IPS should roll this into 4.5!

Sigh...

Edited June 29, 2020 by All Astronauts

[Percival]

Hey there! Works great. Would it be possible to implement it also to the block manager - namely "Post Feed"? Now it loads full posts and uses js truncation. ("post feed" = latest posts from the forum).

Thanks.

[All Astronauts]

That would be an alright tweak. I can probably make that happen. FYI make sure you read the post above yours in this thread 🥴

[CoffeeCake]

On 6/29/2020 at 1:00 AM, All Astronauts said:

Unfortunately, this is a big hitch for search results. If your search term hit in a result is beyond the first 600 character limit, the snippet in the search result will not have any search terms to highlight. If you get lucky and the search terms are in the first 600 characters you still get the highlight of course, but chances are and so forth and etc.

.... 

[All Astronauts]

@Paul E. Yep, I mean, I need to look again to be certain, but I'm at 99.9% certain already (I did - 100%) 

If you have a post that has 1000 characters in it and the last two words are 'blue turtle' and you search for 'blue turtle' that post will of course still be found and displayed in the search results, but the snippet preview is limited to the first 600 characters of the content item in 4.5, so your snippet preview in the results won't have 'blue turtle' in it for the term highlighter to highlight.

It's a hitch I went out of my way to at least partially mitigate by sweeping the content item for at least some term to match on (even skipping common stop words like a, the, and, etc... - hardcoded to English on these initial releases but throwing it open to other langs wasn't an issue going forward), and then taking text before and after that match and pushing THAT out to the results display so you'd usually have something highlighted in each result as a 'match'.

		/* Trim content down to a sensible length for display */
		$content = mb_substr( $content, 0, 600 );

That's what you are getting in 4.5. 

HoWeVeR.... 🥴 .... now that I've looked again that 600 character truncate is on a pleasantly hookable method, so no gloom and doom - sunshine and lollipops all around - I'll get this patched up for 4.5 and we can bask in whatever it is we feel like basking in afterwards.

All y'all watchers but not buyers of this should probably get in while the $6 is still $6. There is going to be a new minimum amount for Marketplace items (if they are not free). 

Hint! It's a two-digit number :)

 

[All Astronauts]

Will look at that post feed block thing mentioned above in a bit. Gonna let IPS IPS on the truncate part; 600 characters it is. They are calling that truncate bit directly via a code call in all the various search result templates now; best leave well enough alone.

Truncating stream descriptions, tweaking image result counts - both all good, along with the now necessary 'find the content that matches the search term and use that bit'. Granted, the find the search term stuff is barebones but gets the job done. Room for improvement? Sure, but that gets in the weeds what with the stemmed term hits coming back and what not.

Anyhoo, this on a content item juuuust slightly more than 600 characters long. <cough>

Never mind the SST - that's a plugin name change that I'm reverting because Super works in place of Speedy just fine. Note the hit is on daemoniac and not just daemon. The aforementioned weeds and so on. 

Exact terms work quite well of course.

[All Astronauts]

@Anawa You are in the clear with 4.5 - as part of their Search/Streams truncate sweep, they've added/changed code in a bunch of places to no longer be pushing out tons and tons of text where it isn't needed. That includes the Post Feeds widget - every item gets cut to 500 characters before display.

FYI I've added an option to swap out the Search the community text on the results page in favor of the actual search term. I usually end up doing this in most of my themes anyways so this was a good time to just flip it into here and be done with it.

[All Astronauts]

Using this plugin as my 4.5 Marketplace guinea pig for the new submission process. So, those of you on 4.5 when you see it in the ACP MP, well, there ya go.

Thought occurred during submission that I should, and will, be merging in my Searchlight plugin (highlight search terms on content items when you click through to them from the search results page) into this Super Search and Stream Results plugin just to save on maintenance. Naturally there will that aforementioned price increase for purchasing, and with this Searchlight add I'll probably be throwing on a renewal fee as this thing starts to grow. Not sure if peeps who buy in prior to renewals added get grandfathered in or not, but just a heads up.

[All Astronauts]

On the fence? Best get in now...

Slightly related: 

That's probably gonna get a bump for 4.5. IPS did knee-cap most of my new buys by building in search tracking, but mine is still more precise and detailed. I doubt that is enough to generate new sales though 😞 - anyways... if any of you search junkies are interested, post up there in the support topic as my gaze will likely be turning in that direction soon.

[All Astronauts]

Waiting on approval, but here's a peak at the new Searchlight coming in SSSR 3

[Steph40]

Good news, thanks for the peak.

More then a peak I think it has been approuved.

[All Astronauts]

Nope. Still version 2 in the MP.

It's kindof annoying right now in that when we submit a new file, if we edit the description to go along with it, the new description is there, but not the new screens or the new file. Those latter 2 don't show up until the new file is approved.

 

Edited November 24, 2020 by All Astronauts

[All Astronauts]

Now it's up.

Note, Gallery is exceptioned out so if a search result is a Gallery thing, Searchlight won't fire there (the page loads and THEN a popup fires with the image and comments and that's just confusing the hell out of everything right now) and those of you on Elastic Search with stemmed search results - meaning you searched for... I dunno... "baconese" and stemmed results give you some with just 'bacon' the results with just bacon might/probably are not highlighted yet. The new and improved Searchlight is doing exact term matching for the moment with markJS. 

We'll get there. This is one of those things I NEED for a site so expert more eventually.

Also, Searchlight is REALLY locked in on only firing when you come out of the search results page. The search terms are only placed into a Searchlight cookie when you click a search result. The longest that cookie will exist is seven seconds which is just a guess on what a crap site might take for loading a page. Regardless, the moment the page actually finishes loading, that cookie is wiped out after I grab the terms with Searchlight. Getting to the point, if you refresh a page that has Searchlight marks on it, they go away - there are no search terms for it to mark any more.

Lastly, this does not mark out hits in content titles anymore. Not much of a need to flag those out - your hit is right there in the title so... Always looked a little dumb anyways.

[All Astronauts]

Not a lot of feedback (any...) for Social Search so looks like that might be dead-ended. I might throw a few things at it and maybe release sometime but with nary a word from purchasers hard to move forward on it.

Anyhoo, this thing.

I've spotted on my sites a few tiny hitches. There are certain cases where a double mark gets applied which messes things up a little. That might take a bit to track down though, but it is on my radar.

What is being addressed right now are cases where matches are not occurring when the word has a punctuation bit directly afterwards when you have things set to "exactly" for matches. And that is happening because I forgot to pull over that option in settings from the old Searchlight plugin - its there in SSSR but someone forgots to add it into the settings form.

Wonder who...

More specifically, with "exactly" set I need to add in additional filters for these ending punctuation marks. Exactly does mean that precisely. "Thing, " is not the same thing as "Thing " so you can see the problem here.

I also am leaning towards REMOVING the on-mark navigation (the arrows that appear when hovering over the mark) as in hindsight its a bit much. With that though, some improvements to the side-page mark navigator, borrowing from the Honey draggable popup thing many of you probably have installed.

Fridays I tend to poke low-key things so this is being worked over now.

[All Astronauts]

Also I saw a squirrel so you know what that means...