Social Search

[All Astronauts]

 

[Joel R]

Does this aggregate or identify popular search terms? 

[All Astronauts]

Not on the initial release. One could easily scan the entries (eye ball them) in the ledger and then do a title search when you spot some regular ones to see how many there actually are. 

[All Astronauts]

@Joel R

It does now...

Social Search 2 Released!

• New statistics feature
• More language keys
• Misc tweaks

 

[jair101]

Do you expect any issues with non-english characters? I have noticed some mods having such limitations. 

[All Astronauts]

Statistics:

So, firstly, this isn't as easy as it appears so I've had to make some reasonable adjustments here and there to, hopefully, capture the bulk of what people are looking for.

Overview Tab:

For now it is just a dump of the dashboard widget information (as not everyone will want the dashboard widget to be the only place to get some stats...) plus a quick one-liner showing what percentage of all searches are tag searches or include tags as a filtering element. Will look to improve this later on.

 

Terms Tab:

This is mostly EXACT MATCHES! That means spelling and formatting counts. There is no "Fuzzy" word and phrase matching going on; honestly that's way beyond the scope of this project for the time being. However, I did want to make an effort to merge together a common instance - that being when someone searches for something without quote, and when they search for it with quotes. For example:

Mass Effect
"Mass Effect"

As it stands right now, those will be two uniquely different searches. And counted separately. What if I remove the double quotes?

Mass Effect
Mass Effect

Now those will both be counted the same. Much better. So for terms, I do that.

 

Words Tab:

This is just an extension of the terms search. For each term, I break it apart into individual words, and then do the same counting on them. There is, naturally, a problem. Letters and common short words. A, the, then, it, its, it's and so on (for English at least). This means I have to parse this stuff through stop words files. Right now I'm forcing English AND Russian stopwords on through. I can, later on, option this stuff out for configuration in the settings but until then, if you have other languages besides English hand Russian, pls give a yell so I can add the stop words json word lists into the routine. I also nuke probably all non-alphanumeric characters here so we are dealing with just "words" and nothing else. 

Tags Tab is pretty much the same as the words tab.

Hey! I forgot to include standardized letter casing during matching on these things - and I should - look for 2.1 real soon, that should tackle the last of the big things here!

[All Astronauts]

6 minutes ago, jair101 said:

Do you expect any issues with non-english characters? I have noticed some mods having such limitations. 

Well it appears to be working on a Russian site so far. Outside of that ??? - I'd need to test or get data back from people using it on their non-English lang site.

[jair101]

34 minutes ago, All Astronauts said:

Well it appears to be working on a Russian site so far. Outside of that ??? - I'd need to test or get data back from people using it on their non-English lang site.

My website also uses cyrillic letters, so I should be fine. Thanks, I'll consider it. 

[SJ77]

EDIT: Really great IPS customization. Thanks  @All Astronauts

Edited November 26, 2018 by SJ77

[All Astronauts]

Social Search 2.1 Released!

• Forced Lowercase Evals.
• Misc. Bug Fixes, etc...

 

This means on the stats pages where before all the following would be recorded separately:

Mass Effect
mass effect
MASS EFFECT
MaSS effCT

Those are all now eval'd as:

mass effect

And will be recorded the same. Keep in mind, there is not (and prob won't be) fuzzy logic matching any time soon (or ever) to catch things like:

massy effect
mass   effect
moss effect

And so on. Can't help things if users add spaces or mis-spell stuff.

Note this forced lowercase is only on the stats pages for matching purposes - all searches remain recorded "as is" and unaltered.

[SJ77]

2 hours ago, All Astronauts said:

And so on. Can't help things if users add spaces or mis-spell stuff.

I love working magic on strings.

I think you could, (and should) also use a trim feature http://php.net/manual/en/function.trim.php to remove spaces at the start and end.

As far as spaces in the middle, if you really want to get creative you could remove all spaces before doing your comparison.

Here are some things to consider solving for.

• Same search different order. "mix together" vs "together mix" .. it's the same search
• deal with "S", "ED" or "ING" on the end of strings and remove them before comparing. "Water bottle" and "water bottles" are the same search.
• Remove all dashes and special characters "red-car" vs "red car"

That's just some ideas for now.

This is GREAT,... keep it up! 🙂

Edited November 27, 2018 by SJ77

[All Astronauts]

6 minutes ago, SJ77 said:

I think you could, (and should) also use a trim feature http://php.net/manual/en/function.trim.php to remove spaces at the start and end.

I could, the number of times a search comes in with opening or ending spaces prob close to nil but its an easy add.

7 minutes ago, SJ77 said:

Remove all dashes and special characters "red-car" vs "red car"

That should be happening on the word tab already, but it becomes a question of intent when comparing full terms. 

9 minutes ago, SJ77 said:

deal with "S", "ED" or "ING" on the end of strings and remove them before comparing. "Water bottle" and "water bottles" are the same search.

Ditto on intent.

I'm going let this sit for a while, let users get some data built up and see what shakes out but keep the suggestions coming (provided I don;t need to fully implement a massive text evaluation engine...) 🙂

[SJ77]

Well, I got around to trying this today. It's pretty awesome.

One thing I was expecting that it doesn't do (yet?)  would be a widget that shows what is trending

So instead of just recent searches, it could show common searches and rank them to get the top 5 trending thing folks are searching for.

Just a random idea.

[All Astronauts]

Yeah, I can just pull some of the code from the stats page and stick it in a widget. Won't be as up-to-date as the stats in the acp, I'll need to task this to prevent melting the server, but totally doable.

[All Astronauts]

Version 3 Released!

• New Popular Search Terms Widget
• More Language Keys
• Misc bug fixes

Heads up those of you with non-English languages: let me know if you run into any bugs with the new popular terms widget. The way I store and retrieve that widget's data might error on UTF8 special characters.

Edited December 9, 2018 by All Astronauts

[SJ77]

On 12/2/2018 at 11:09 AM, All Astronauts said:

Version 3 Released!

• New Popular Search Terms Widget
• More Language Keys
• Misc bug fixes

Heads up those of you with non-English languages: let me now if you run into any bugs with the new popular terms widget. The way I store and retrieve that widget's data might error on UTF8 special characters.

Ever since this change I get the tasks locking frequently error in the ACP.

[All Astronauts]

Odd. There are two tasks with Social Search. One runs every two days (and all this one does is trim the search log based on your settings - 90 days, never, etc.) - it's literally a single database delete line, the other every ten minutes and is a trimmed down version of of the dashboard/stats stuff which then stores info in settings - loading a topic has way more going on than this little thing. On a site of my own where this is installed everything is fine, no stalled tasks or anything.

Does the frequently locked error state which tasks are locking?

Edited December 6, 2018 by All Astronauts

[SJ77]

I haven't seen any errors. I have removed the widget for popular searches from my home page.  I will wait and see if I still get the locking error.

Removing the widget should end the task, correct?

[All Astronauts]

No. The task is not bound to the widget. Also, when you get that locked tasks message in the acp, at the end it will list which tasks are locking. If you can give me acp breifly, I can look at the database directly to see which ones are locking or shoot me a PM and I can walk you through it.

Edited December 6, 2018 by All Astronauts

[All Astronauts]

Social Search 3.2 Released!

• Minor fix for edge case when search is run with absolutely no search parameters (makes sure to not record it)

[Millipede]

Re the coloured tags for search wall:
- Is there a way to reset colours when testing variations? Eg: changing colours in the settings only seems to affect new searches, not previous ones?
- How long should it be before any colour changes take place? It seems to be at least 24 hrs...

Could “Search Ledger” be the top menu, please, so that when clicking on the ACP menu it takes you to the ledger instead of to settings?

Might there be any possibility to disable search terms by admin? I often search for a topic for admin reasons and not out of interest. Having to keep deleting these in ACP is a bit of a pain (especially as there aren’t checkboxes to choose multiple entries...).

[All Astronauts]

1) Colors are generated at the time the search is stored, no way to change them (at the moment) - reason being is this is mainly for the search wall and after awhile the searches will drop off, so any color changes would just be temporary, as the new searches with the new color palette will then be displayed, etc...

I may be able to perhaps give you a live look at changes? In other words, you change settings and it gives a preview of what 10 random colors under those settings would look like. I'll give that some thought.

That or I could set a routine I suppose to sweep through and change... I dunno, last 30 days back of searches when you change colors, or at least an option to do so. IF people are storing searches forever, and it's a heavily trafficked and searched site, updates like this (for the whole thing) could be brutal.

2) Color changes are immediate for all new searches going forward. No waiting or delay.

3) Yes, will make that change today. Also had to mix up the dashboard widget for scrunched formatting, and the widget now has links to ledger and stats.

4) Option for admins to not have searches stored? I can do that.

[Millipede]

1) A preview of colour changes would be a great idea!

2) Your answer to 1) explained the time it takes; mine's a quiet forum, so that's why it took a while to show any changes.

4) That would be super handy too, thanks

 

[All Astronauts]

Social Search 4.0 Released!

• Revamped dashboard widget.
• Moved settings menu option to bottom, below ledger and statistics.
• New option to NOT store searches made by administrators.

[Millipede]

Wow, that was extra super quick for those new features, thanks!

Store searches by admins: isn’t storing searches by admins, regardless of whether it’s enabled or disabled. That’s fine by me, as it’s what I asked for , but is it maybe a bug that it's not storing admin searches at all?