Searching needs significant improvement.

[Makoto]

There's plenty of room for improvement in IP.Boards searching function.

All in all, it's always been a pain for me to use. It's almost always easier to simply use a Google search on my site to find what I want. However, this is only because my site is well indexed by Google. Not everyone is so lucky. In this thread, I'm going to go through some of the problems I've had, in no specific order.

My memory is fairly sub-par, so I'll likely recall a few other complaints and suggestions over time that I'll add to this list, but this is simply a thread to get a discussion started. A thread to gather feedback and gather comments and further suggestions from the rest of the community. A thread to hopefully drive more attention to improving one of the most basic and most used functionalities of many forums, searching.

---

Relevance

Have you ever tried to search for an app in the marketplace using the built in search function here? Let's give an example. Try and search for this app now: It's called "Links Directory," so you'd search for just that, right? Links Directory. Huh. It's not there. The title of the app is clearly "Links Directory," but it's nowhere to be found on the first page. Why? It seems to be simply because the search results are organized purely by the submission date, relevance has no factor here. The actual app is third to last on the search results page. A quick Google makes the page significantly easier to find. directory It comes up as the very first result.

http://community.inv...inks-directory/







https://www.google.c...links

---

Searching by thread, not post author

One thing I find rather annoying is that I cannot search by a threads author. You can only search by post author. So, say you want to find a thread started by another user.

The only way you can go about doing this is searching by the post author. This will include all threads that the user has posted in. There is currently no way to search only by the threads author alone. A simple extra functionality that I'm interested in hearing others opinions on.

This could be useful when trying to search for a story a user posted, a guide a user wrote, so on and so forth.

---

Searching by multiple tags

Currently you can search for multiple tags at once, but this isn't exactly what I mean. If you search for an image with the tags "blue" and "nature", as an example, it will return all images from the gallery with either the tag "blue" or "nature". Unlike with keywords where you can define to search for "all words" or "any words", you cannot search for posts that contain "all tags" or "any tags."

This is a functionality that I would personally find extremely useful in my community, especially in the gallery and downloads system. When you have a massive amount of images or posts in your community to weed through, being restricted to searching by a single tag at a time can be very inefficient. When you have a community that makes intensive use of the tagging system, why should you be restricted to only searching by one tag at a time? Tags can be used to describe many things, especially in images, and this is why many popular image hosting platforms (such as Gelbooru) make use of this basic feature.

My suggestion here is simple. Give the option to search for either "all tags" or "any tags," just as you can with keywords.

---

You can't search archived and live posts at the same time

. The technicalities of the problem are understandable, and it's more or less considered a "trade-off" for using the archiving system by most. Apparently, with some configuring, you might be able to get Sphinx to search in both tables.

This is a functionality issue I've

posted about before

This also seems to be an issue already acknowledged by developers, but I'm simply posting it here for the sake of including everything I can think of in my list.

---

(Bug) Searching by title and content in archives only searches by content.

This is a bug and a problem I've just happened to come across. When searching by "title and content" for archived posts, it only searches by content. It does not search by thread titles. See the following example. I'm searching for posts with the keyword "Konata" that are made by the user "Shu" in my site's Fanfiction forum. This is what is returned when searching by title and content.


Only one result. There should be at least several. This baffled and frustrated me for a while, until I happened to figure out the cause. Now, if we go back and search by content only, we'll get the same thing. But, if we go back and search by title only, see for yourself.


The most logical conclusion seems to be that the search function refuses to search by both the title and content, even though this is the default option. Try it yourself. Go search for something generic in your forum archives. Search by both title and content, title only, and content only. See some inconsistencies with the results? This is just one of the problems the archiving system brings. It's a good idea, but there are a number of bugs and problems I've seen because of it, to the point I'm considering simply disabling it and unarchiving all posts on my board at this stage. It's still a relatively new feature though, so I can't really complain all that much about it.

)

(

Bug report

[esquire]

These are excellent points. Very well done.

While these are certainly necessary, fortunately the system doesn't seem broken and just needs some tweaks. But they do need to get done.

[Rimi]

Regarding your example of relevance, I've notice that the marketplace issue is due to the fact that the words you search for are included in the description of various marketplace downloads and marketplace results are sorted by submission date. This is what the problem is. Something needs to be done to either attribute more importance to the title of the file and thus ranking it higher or the default search (because you can indeed change this in advanced search) needs to be changed to the title only.

I'm not exactly sure what you mean by searching by thread author..but doesn't this accomplisht that?
[searchInKey]=&userMode=title"]Link

If that's not it then you can search by author and only in thread titles in the advanced search too...so I guess I'm not sure what you're asking here?

Searching in the tagging system would be best off it it was completely overhauled and was just like the booru system that I've seen you refer to so often. Basically I agree with you.

[Makoto]

Regarding your example of relevance, I've notice that the marketplace issue is due to the fact that the words you search for are included in the description of various marketplace downloads and marketplace results are sorted by submission date. This is what the problem is. Something needs to be done to either attribute more importance to the title of the file and thus ranking it higher or the default search (because you can indeed change this in advanced search) needs to be changed to the title only.

Even when searching by "Title," it only sorts it alphabetically. There's no way that I know of to have it simply display by "best match."

I'm not exactly sure what you mean by searching by thread author..but doesn't this accomplisht that?

Link

If that's not it then you can search by author and only in thread titles in the advanced search too...so I guess I'm not sure what you're asking here?

Basically say you have a forum where one of the main sections is entirely dedicated to publishing stories/fanfiction. A section for authors, basically. Now say you have a common subject/series of fiction on your forum, a show many authors on your board use for fanfiction, as an example.

If you wanted to search for all stories published by a single user from the "Lucky Star" anime series, you would do a keyword search for "Lucky Star" and search for all posts made by this author. The problem with this is that it will also return all threads started by other members that this author has simply commented on. It may seem like a small thing and it may not be worth actually going through the trouble of adding extra clutter to the search page, but I was just listing off the top of my head. I'm having second thoughts about the suggestion now.

If you search by title only, does it actually only pull up threads started by the author you search for? I honestly never even considered trying this before, but it would make sense I suppose.

Searching in the tagging system would be best off it it was completely overhauled and was just like the booru system that I've seen you refer to so often. Basically I agree with you.

Glad you agree!

[Rimi]

If you search by title only, does it actually only pull up threads started by the author you search for? I honestly never even considered trying this before, but it would make sense I suppose.

...huh. I actually don't know. I mean...I would assume so. But if not then yeah I see what you're talking about now. I can see the argument for it not being the most useful thing in the world, but it does have it's uses. Maybe another field in the advanced search that says "Topic Starter". I think that would work.

[miraclesun]

Yes. Yes. Yes.

[Makoto]

I've discovered another bug when attempting to search by post author alone.

This is related to what Lucy was talking about, I believe.

If you search a specific forum for posts made by a specific member and by title only, using only the author name and no addition keywords, it will return results that.. well, plain and simply don't make sense.

Say I'm searching for posts made by the member Bob in the Fiction & Fanfiction forum on my site. I'm searching by titles only, not titles and content.

Upon submitting the search query, you are returned results from all forums on your site, not just the Fiction & Fanfiction system. You are also returned random threads that the member has not started nor ever replied in.

If you do include keywords with the search query, it seems to work just as Lucy suggested. It returns threads the user you are searching for has started, but not the ones they've simply replied in.

[esquire]

I just searched for "prefix" and "prefixes" and I couldn't find the mod I was looking for that I know exists. Maybe I'm searching incorrectly but most people would go to the searchbox, type in that term and expect to see it. Perhaps parts of the forum are segregated or not searched such as the downloads modules?

[TSP]

Another example of area where the search needs to incorporate some sort of logical sorting is the members search.

Just to give you an idea of how useless it is.

I start of by searching for Mark, because I want to find the user that's named exactly "Mark", nothing more, just Mark.


What I get when I submit search is 60 - SIXTY (yes, 60) - pages of results. And Mark is not to be found anywhere near the top. I also browsed through the first 20 pages unable to find him.



Believe it or not, but this is not the person I'm looking for...


I could of course change the sorting to sort by title, but that doesn't help much either. But makes it "sligthly" more easier to find him. Finally found him on page 37 sorting by title.



I'll have more luck browsing the forum, finding a post from him and go to his profile from there or wait for him to show in the online list, remembering his signature so I can search for a phrase from that. Or just google him:



So what would be a logical way to sort it? Well it's difficult to give a clear response on this. But for the most part, when searching for members, you search for the member that match the name you write in, exactly.

So this is what I think would be the best way to sort it.

1. The user having a display name that exactly matches "Mark".
2. Users having a display name that starts with "mark"
3. Users having a display name that ends with "mark"
4. Users where the match for "mark" was found in their signature or whatever

So an example could be:

Mark

Mark.

Mark A

Mark John

Markeh

My Name is Mark

IAmMark

Some random guy with the search key in his signature or whatever

[EarthDog]

I could't agree more on those thoughts.

Especially multiple tag searching. I am surprised that its not standard. It kind of negates the tags capability ....

+1

[Eric Allione]

I'm not sure where I'm supposed to "sign" but I have always been disappointed with IPB's search feature, especially when forums choose to prevent 3-letter word searches (which is annoying and something I've never understood).

If I want to find something on these forums I just type a keyword into Google followed by community.invisionpower.com. I saw a hook that added a Google search as an alternative but I don't think it works on the current version. Upgrading the search feature would directly improve the quality of posts (less repetition; more informed questions).

But sine I'm sure that competing with the Google algorithm would be a waste of resources and near impossible, just having an embedded Google search as a default feature would be nice.

[EarthDog]

especially when forums choose to prevent 3-letter word searches (which is annoying and something I've never understood).

To be honest this is a restriction of MySQL and can be solved if you persuade your hosting company to change this setting in your server. But this is unlikely to happen if you are sharing hosts as it would increase server load on queries...

[Makoto]

I've read IPS making public statements that they have had plans on improving the search engine, but I'm sure we'd all appreciate some amount of recognition in this thread.

Have you applied any of the suggestions made here already? Are you looking to apply any of these suggestions? What changes are you planning to implement in the future and how far off in the future do you think the majority of these changes will be?

[Rimi]

I've read IPS making public statements that they have had plans on improving the search engine, but I'm sure we'd all appreciate some amount of recognition in this thread.



Have you applied any of the suggestions made here already? Are you looking to apply any of these suggestions? What changes are you planning to implement in the future and how far off in the future do you think the majority of these changes will be?

Based on statements they've made I'd say that they are currently planning out how to go about revamping the search process. I know for sure that they haven't started yet. So now is the perfect time to tell them what you want.



Don't you know IPS by now? They don't announce what they're doing/what they've done until it's 100% done to their satisfaction.

[Makoto]

I've posted several of my suggestsions here.

[EarthDog]

I have faith on the guys, i am sure they are gonna deal with it...After all it affects an important function on the suite...

[bfarber]

Searching will be rewritten and overhauled in the 4.0 Suite. I'm afraid we can't release any further information than that at this time. We are still planning 4.0, after all.

[Heyhoe]

Good to hear, I wanted to find Member Map in the marketplace and found it a few pages down!

Search has definitely needed more attention for years.

EDIT: Actually, it is on the first page, but when searching by "Relevency" It's half way down the page... Even though the search term is an EXACT match.

[bfarber]

Unfortunately, that's simply MySQL's interpretation of what is relevant. There is only so much that can be done about that, although we've looked into some ways to weight MySQL's relevancy differently.

[GreenLinks]

First time i ever heard someone who is complaining about Sphinx relevancy. MySQL Full text will never reach Sphinx relevancy levels. If you are having issues on relevancy , i highly suggest you to read more on Sphinx Documentation.

[Makoto]

First time i ever heard someone who is complaining about Sphinx relevancy. MySQL Full text will never reach Sphinx relevancy levels. If you are having issues on relevancy , i highly suggest you to read more on Sphinx Documentation.

Sorry.. who was complaining about Sphinx's search relevancy again?

[miraclesun]

Searching will be rewritten and overhauled in the 4.0 Suite. I'm afraid we can't release any further information than that at this time. We are still planning 4.0, after all.

yaaaaaaas!!

[Makoto]

I'd like to throw a potential idea out here for my suggestion regarding searching by multiple tags.

I believe a combination of the current system with something similar to what a system such as Danbooru or Gelbooru uses could be very useful for "advanced users."

The only real problem with this idea is that it would require your members to learn how to use this advanced searching technique if they want to take advantage of it.

You could consider it somewhat similar to advanced searching on Google. My idea is basically this:

• By default, when searching by tags, it works as it currently is. It will match any of the given tags.
• When searching by tags with a + operator, it will only return results that match all of these given tags.
• When searching by tags with a - operator, it will only return results that have none of these tags.
• All three of the above can be combined in a single search query.

To offer some examples, a user could search for the following tags:

+nature, +blue, -park

This would return all results that match both the tags "blue" and "nature", but would not include any results that had the tag "park".

This is a pretty generic and bland example, but it still shows the basic idea of things.

What's different about my idea is that it adds in a bit of extra control and complexity. Another example,

+nature, -park, blue, green

Still not the best example, but this would do mostly the same as above. The difference is it would also only return results that have either of the tags blue or green with it. So, for example, it could match an image with the tags "green, nature", "blue, nature", or "blue, green, nature" all in the same search query.

This may seem a bit overwhelming and overly complex, but I just figured I'd write my idea down here, since I might try and add in a similar system to my forum.

Still, even if it is complex, I think adding in some type of system that would at least allow members to search tags with a "match all" option would be very useful.

I've had many requests for this on my forum, as well as requests for the ability to block/ignore specific tags.

[esquire]

Great suggestions Kirito. My suggestions go beyond just syntax but the desperate need for drilldown in advanced search, e.g. searching in the main UI to allow searches on in specific profile fields. For example, how do I find all users who are from Idaho and have checked off in the "favorite food" attribute that they like potato chips? In IPB you can collect a good amount of data but no front end tool that makes it easy for users to search through all that great data you've collected.

[Makoto]

Great suggestions Kirito. My suggestions go beyond just syntax but the desperate need for drilldown in advanced search, e.g. searching in the main UI to allow searches on in specific profile fields. For example, how do I find all users who are from Idaho and have checked off in the "favorite food" attribute that they like potato chips? In IPB you can collect a good amount of data but no front end tool that makes it easy for users to search through all that great data you've collected.

The members app lets you do most of that, doesn't it?

http://community.invisionpower.com/members/#filters