Searching on Forums is totally unreliable

[Ronald Epstein]

I know you guys are working on an overhaul of IPS for version 4.0. One of the things I hope you are working on is optimizing the forum search engine so that it can actually find things you are looking for.

Here are two examples from users of the forum who are complaining about not being able to find what they are looking for even though the search terms that were entered were very clear:

Quote

I was looking for information on the movie, THE FLY. I knew there was a thread on the forum with that title. However you can see here how I had to really play with the search results to find that thread...


1. Go to the Blu-ray forum
2. Click on the SEARCH box for: The Fly (it will default to searching the forum)

Nothing should come up. Absolutely nothing.

3. Now repeat step #1
4. This time do a forum search for: TCFHE Press Release: The Fly

Your results will show THE FLY in the list. Not first, mind you, but it will be on that page.

Here is another instance:

Quote

I am simply trying to find any and all threads in the After Hours Lounge which have anything to do with dogs.

So I head to the search engine (both simple and advanced) and attempt to enter 'dog" into the search field. I try "just titles", "titles andcontent". I try to just search AHL or the entire forum.

And all I ever get is: No results found for 'dog'.

Obviously there have been a number of threads about dogs in the forum. In fact, there is one new one there right now. I know, because I want to move the posts into a pre-existing thread on the same subject...BUT I CANNOT FIND IT!

Update: I found the threads I was looking for--the old-fashioned way. By going to the forum's index page and going back, back, back chronologically until i found the various threads.

And I did what I wanted to do--which is to move the posts from the new thread into one of the old ones...as well as include links to some other threads on the same topic.

For the record, Ron (when you communicate with the IP.Board folks)...all of the threads I was looking for had the word "dog" in the title. While one "dog" has an exclamation point at the end, the others were unencumbered parts of the title ("My dog has been put to sleep" and "At what point do you choose to put your dog down?" and "I've Got a New Dog!").

That is a major "FAIL" on the part of the search engine here.

[Andrej]

>It has been mentioned.

[theblackelk]

Could it be since those terms are only three letters? Did you change your minimum search word length in the ACP?

The default minimum would be four. Three is much better, in my opinion.

[Rheddy]

It's because of the horrible way that the forums is setup. I've never really unuderstood why IPS felt it necessary to limit searching so that it disregards anything with 3 letters, 2 letters and so forth. While you can set up how many letters to disregard, I think it limits the search.

[Rhett]

Mysql by default is limited to 4 letters, it's not an IPS limitation it's a mysql limitation. However as Brandon has mentioned in the other topics on this, search is going to be a major focus in the 4.x series.

If you are running your own mysql server you can adjust it back to 3 characters as well, (ft_min_word_len=3) however this wouldn't be wise on a large forum in most cases and why mysql defaults to 4 char.

[CalendarOfUpdates]

It's because of the horrible way that the forums is setup. I've never really unuderstood why IPS felt it necessary to limit searching so that it disregards anything with 3 letters, 2 letters and so forth. While you can set up how many letters to disregard, I think it limits the search.

Um, because the default minimum for MySQL full-text search was set to four characters for years. When Version 5.6.4 was released (I believe) it was changed to 3 characters. They are still supporting MySQL 4 (suggesting MySQL 5 though) for the 3.x series hence the four character limit out of the box.

[Ronald Epstein]

Thank you for the replies.

Lets hope IPS gets this fixed in the upcoming version 4

[Michael]

As most folks said in this topic, it's not an issue that's due to IP.Board, it is a MySQL limitation. Thus, there's not much of anything IPS can do to 'fix' this for 4.0 apart from providing clear instructions on how to do things like tweak your MySQL configuration or how to use alternate search engines like Sphinx (both of which they already do).

[GreenLinks]

As most folks said in this topic, it's not an issue that's due to IP.Board, it is a MySQL limitation. Thus, there's not much of anything IPS can do to 'fix' this for 4.0 apart from providing clear instructions on how to do things like tweak your MySQL configuration or how to use alternate search engines like Sphinx (both of which they already do).

This is highly possible the main reason why IPB is falling behind

This is directly an IPB issue. MySQL may have limitations but there are tons of things IPB could do to resolve this issue as starting with developing an advanced Sphinx or other Search Engine integrations.

[Michael]

This is directly an IPB issue. MySQL may have limitations but there are tons of things IPB could do to resolve this issue as starting with developing an advanced Sphinx or other Search Engine integrations.

If the issue is that you can't search for something with three letters, and the Sphinx search IP.Board provides fixes that, then what is the need for some 'advanced' Sphinx search? And even without Sphinx in the picture at all, this can be taken care of with the same MySQL tweak that's been around for years.

OK, if there are other search engine integrations that could be added, great, every other new option helps, but there are already two readily workable options now. Perhaps you could share some other examples of these other search engines that could be integrated? But I doubt there are too many folks who need three letter searching and can't use one of the existing options.

[bfarber]

We already feature sphinx integration which supports 3 characters (by default).

You can already change your ACP configuration such that you perform a "WHERE column LIKE '%term%'" type search. It is very much not recommended to do so, as fulltext searching is far more efficient than the older style of searching, however it is an option out of the box. You can perform three character searches without changing MySQL's configuration using this method, however if you have a large site it is going to be very inefficient.

Finally, MySQL limits your fulltext words to 4 characters or more. To overcome this, the only option is to change your MySQL configuration. This has been the case for many years, although someone above pointed out that MySQL is changing their default in a future version so that may change in time.

[JayX]

To be honest, because IPB is so easily parseable by Google, if your forum is open to the public I find it's easier to just open a Google search for (using my site as an example)

site:southwalesmassive.com dog

It does a pretty great job.

[lordi]

To be honest, because IPB is so easily parseable by Google, if your forum is open to the public I find it's easier to just open a Google search for (using my site as an example)

site:southwalesmassive.com dog

It does a pretty great job.

+1, :thumbsup: :tongue:

much easier searching via google

[Fedor Sayapin]

ver. 4  search limit stiil 4 letters. Is this setting in mysql server or in IPS engine? If in IPS could you please point in which file it?

[Paul.F]

It's a mysql thing, nothing to do with IPS

http://www.scriptscoop.net/t/188ed5f76fdd/mysql-fulltext-search-using-2-character-word.html

http://dev.mysql.com/doc/refman/5.7/en/fulltext-fine-tuning.html

The magic of google.