why is search so bad?

[sasiko]

for many users they want to search for stuff but cant completely remember the name so they type incomplete names example

a person want to find a topic called ABC DEF but ends up only remembering and searching ABC DE and that would yield 0 results if you have the search phrase to "and"

sphinx never had that issue, it would had resulted the precise result of ABC DEF even though he searched for incomplete name of either both words even though search type was set to "and"

[Lindy]

The search defaults to exact match. If you want partial searches, you can simply do a wildcard asterisk. Example... Invis* would yield results for Invision, invisible, etc. 

 

[RevengeFNF]

@Lindy and if i search for "Tests" and i expect it to show results for "testing", "tested", "test", etc etc?

This is Stemming Search, where it reduces words to their word stem, base or root.

Sphinx and Elasticsearch supports this, Invision Mysql Search does not...

Sphinx even supports Lemmatization, where for example, if you search for "Run", it will also return results for "Ran"

 

[Dreamlander]

I agree with maidos, the search must be improved.

Now days many engines works default, with Stemming Search,with no  wildcard asterisk added,the users don't know this and it's risky losing content on search,like RevengeFNF said.

The search engine is bad since the old versions ,it is outdated you must do something about it.I thought in the new version was upgraded also.It's so shame.

[sasiko]

On 4/24/2016 at 5:25 AM, Lindy said:

The search defaults to exact match. If you want partial searches, you can simply do a wildcard asterisk. Example... Invis* would yield results for Invision, invisible, etc. 

 

You assume all my members would know * ?

its really caused great inconvienced with your removal of sphinx. i wished i could had reverted back to when sphinx was fully supported.

[chilihead]

We got rid of the antiquated bbcode but we still have the exact or * wildcard search, which I didn't even know you could do. Search needs to be modernized.

Prime example:

Search vbulletin converter in Downloads.

https://invisionpower.com/search/?&q=vbulletin%20converter&type=downloads_file

2 results and none of them are the IPS converter, one is 2013, one is 2014.

Why? Because the file is converters.

https://invisionpower.com/search/?q=vbulletin+converters&type=downloads_file

So, search vbulletin converters or vbulletin converter* and you get it. It should be smart enough to give it to you the first time, and since most users are looking for a single converter, in this example, IPS is losing business. Especially since the IPS Migration page under "Get it now" still has the 3.4 converter  so you have to search for the new one. Also you call it "Migration Tool" on the company page, and when you click to get it it's called "Converters" which makes it hard to find as well in search for those looking for the migration tool. Finding the actual v4 converters was a task but I finally found them.

Short story, search needs improvement.

[Simon Woods]

Just when I thought the problems on my site with search might have been due to it being a new installation or something else like that...

[SJ77]

I agree, search is really hard to use. I know the request is "provide a specific example".  I can just tell you it doesn't feel like it finds anything I want when I am looking for things. I don't like exact searches. Especially because I don't know "EXACTLY" what I am searching for, thus why I am using a search.  I would much prefer fuzzy find. Google search is fantastic, it's almost better to bring in a site google search at this point.

[Lindy]

We have an internal discussion on search. I'll update when we have more info.

[AtariAge]

A competent search system is very important to me.  When you have millions of posts in your forum spanning over a decade, it's really important that people be able to use the search system to quickly find the content they are interested in.  I consider it one of the most important aspects of the forum, really.  If guests cannot find what they are looking for, they may leave and go search elsewhere.   If existing users cannot find what they are searching for, they may start a new topi duplicating existing topics.  The stronger the search system and it's ability to find relevant posts, the better the community, in my opinion. 

It's been frustrating for me to use the search here.  I very often have to resort to Google to perform a search of these forums.  It also frustrates me that search functionality seems to be a second-class citizen at Invision.  I am currently running Sphinx, and I'm concerned that using MySQL's built-in search is going to result in poorer results, and perhaps be considerably more resource intensive on my server versus Sphinx.  This talk of a possible cloud-based solution isn't really the direction I was hoping for. 

Please Invision, give me confidence that my transition from 3.4.x to 4.x will not result in a step backwards as far as searching goes. 

[chilihead]

I'd like to be able to search 4.2 or 4.1.12 and find topics on what is up and coming in that version but it yields 0 results. Probably a mysql limitation (the .) but severely limiting for any software or technology search. I too use google site search but that does not work in private forums.

[AtariAge]

2 minutes ago, chilihead said:

I too use google site search but that does not work in private forums.

That's a good point, I have several forums that are invisible to guests (and Google) on my site, and those forums contain over 600K posts.  

[Joel R]

I remember in another discussion about search that someone made the astute observation that it's not that IPS search is bad, it's that Google's search is so, so good ... and we as regular consumers take the Google search for granted with their thousands of PhD data scientists constantly fine-tuning the algorithms and research behind search.

IPS' strength is not in developing search.  But everyday users and admins expect the level of Google search.

 

[sasiko]

1 minute ago, Joel R said:

I remember in another discussion about search that someone made the astute observation that it's not that IPS search is bad, it's that Google's search is so, so good ... and we as regular consumers take the Google search for granted with their thousands of PhD data scientists constantly fine-tuning the algorithms and research behind search.

IPS' strength is not in developing search.  But everyday users and admins expect the level of Google search.

 

no, sphinx and elastic search outperform ips default search so many times. 

read RevengeFNF posts here for the very reason

[RevengeFNF]

@Joel R that's why there exists software like Sphinx or Elasticsearch. Invision doesn't need to reinvent the wheel, they just need to take advantages of the software that do that. They already use 3º party software in Invision Suit instead of reinventing all.

Google only have access to the HTML of our site, but we have access to our own Database, so we have the responsibility of having a better search than Google.

There is more advantages in Sphinx than only the better search. For example, my site have 1.6 Million posts(its small compared to others), and my Search Index have 1Gb. That means it will use 1Gb of Ram in the Innodb Buffer. With Sphinx it used less than 100Mb of Ram, with the advantage of being faster. But the big feature of Sphinx is that it supports Stemming and Lemmatization Search, although Invision never supported it in 3.x and 4.0.

I don't like to talk about Invision competitors, because i think Invision is superior(that's why i am faithful to them), but in terms of Search, Xenforo is superior. They support Elasticsearch and they support Stemming Search.

Also, we don't need Cloud Search for this, when there are free software that can do the same. We don't need to pay more money for that... But i understand that Invision also wants to add supports for Cloud Search for big and busy sites. For the majority of their clients, i believe Sphinx or Elasticsearch is more than enough. 

[Lindy]

RevengeFNF - I promise, I hear you loud and clear, my friend... you want Sphinx.  Sphinx isn't likely in the cards - we think there's better options, but Elasticsearch is something we are looking at.

The information in this topic has helped us better understand the challenges and concerns, both within the core product and alternative searches. 

[RevengeFNF]

11 minutes ago, Lindy said:

RevengeFNF - I promise, I hear you loud and clear, my friend... you want Sphinx.  Sphinx isn't likely in the cards - we think there's better options, but Elasticsearch is something we are looking at.

The information in this topic has helped us better understand the challenges and concerns, both within the core product and alternative searches. 

Lindy, Sphinx or Elasticsearch, i don't care. I just prefer a good solution that don't involve the need to waste more money.

[sudo]

Elasticsearch would allow Cloud based for those on AWS and enable users with their own resources to gain full search.

[Haku2]

16 hours ago, Lindy said:

RevengeFNF - I promise, I hear you loud and clear, my friend... you want Sphinx.  Sphinx isn't likely in the cards - we think there's better options, but Elasticsearch is something we are looking at.

The information in this topic has helped us better understand the challenges and concerns, both within the core product and alternative searches. 

Hi @Lindy,

I'm more comfortable with Sphinx, but whatever works best would be preferred as long as my community doesn't need to rely on any external third-party services or resources for the Sphinx-alternative to work.

Kind Regards,
Haku

[Lab Rats Rule]

I was breezing past this and had to comment,  Search for sharing in the admin CP, nothing comes up.  You have to type sha, shar, or share to bring up the "sharing" settings.  I encounter this weird issue often with words.

[Morrigan]

@Lab Rats Rule That's a completely different issue than what is being discussed here. The ACP search is based on keywords in the ACP. You may want to report that as a bug.

 

What the members are talking about here is exclusively the front end search capability.

[Dylan Riggs]

I really don't care how all the search stuff works, all I know is, I have better luck going to google to find what I need on these forums better than I do actually using the search function ON the forums itself.

How all that stuff works behind the scenes is beyond me sometimes. All I know is as everyone has stated numerous times. They can try to do a small search for something and usually I'm returned something TOTALLY different than what I'm actually looking for, or I'm returned with a 1000 mods from adriano lol.

[sasiko]

im revisiting this topic after realising i get 0 search results here when isearch for x-sendfile anywhere and using both/or phrases

can ips staff explain why x-sendfile gives u no results? 

 

 

and i had to find this place by searching google and it was the first result too

 

im still hoping we get elastic search implemented anytime soon since ips search results is just too underwhelming

[RevengeFNF]

If you search for "*sendfile" it works. Which means its indexed.  

Now imagine that my site is in Portuguese, and we have many words with hyphen "-".

[sasiko]

2 hours ago, RevengeFNF said:

If you search for "*sendfile" it works. Which means its indexed.  

Now imagine that my site is in Portuguese, and we have many words with hyphen "-".

yeah that is a horrible situation for you. but ips suffers from other symbols too aswell?  i mean if keyword includes any of the symbols it yields 0 results

So do you implement just a google searchbar on your site to replace with ips own search function?