Jump to content
Josh
 Share


IP.Board 3: Global Search

During the initial design phase for IP.Board 3, one of the first areas that we identified for a major overhaul was the search system. In IP.Board 2, each application is required to have it's own search engine, which creates many silo's of data that can not be easily searched. IP.Board 3 will introduce a new global search system that will make all of the content of your community easily searchable, no matter where that content is located. You will have the option of showing the results from all applications within one listing, or filtering your results by application.

One of our primary goals in IP.Board 3 is to dramatically increase the integration level between all of our community products, so that they feel more cohesive than in IP.Board 2. This new search system is just one example of how we are working to achieve this. However, we don't want this ease of integration to apply only to our own products, so modification authors will be able to utilize this same system for their own applications. Through the use of a simple plugin system, modification authors will be able to have content from their applications show up within the global search results.

The new search system will also be more streamlined and easier to use. The new "Advanced Search" form is much friendlier than the form in IP.Board 2 is, making it much easier to determine how to formulate your search. Additionally, every page will include a quick search box in the header that you can always use to search, no matter where in the system you are. This quick search box will also include 'live search results' that will dynamically show you a preview of your results as you type the search term in the quick search box. The live search will be context-sensitive. That is to say, if you are in a forum, it will search that forum; if you are in a topic, it will search that topic; if you are in the gallery, it will search the gallery; and so on. It provides a link to view all search results (from all applications) should you require it. Additionally, there is a setting to disable the live search should you not wish to support it on your board.

Performance has also been an issue with our current search system, our new system aims to overcome this in a variety of ways. The first is by creating a global search index, which is what powers all the features that I've mentioned previously. This global search index is much easier and quicker to search than searching the post table. Since searches will no longer be performed directly on the post table, there will no longer be any issues with locking that table during a search. To reduce the disk space overhead and make searches even quicker, the search index has a stripped down version of the content that contains no bbcode or markup of any kind.

The second way we will be improving search performance is by supporting Sphinx out of the box. Though you will need to install Sphinx yourself, once you have done so, enabling it in IP.Board 3 will be as easy as changing a setting in the ACP. You will be able to remove the full text index on the post table using either the new search index or Sphinx, which will dramatically reduce the size of that table.

The new search index will have the added benefit of enhancing other areas of the forum as well, such as the "View new posts" feature. View new posts will now include any kind of content that is searchable, so you will see forum posts, gallery images, blog posts, etc in the new post listing. There are some other enhancements as well, but we're going to save those for a future blog post.

We're very excited about these changes and we hope that you will be too

 Share

Comments

Recommended Comments



This is a big improvement, and the possibility to search in plugin data is awesome!

One quite important thing is the handling of three-character words/abbrevations. Out of the box, it is currently not possible to search for that short words. I'd appreciate the possibility in IPB3. Many thanks!

Link to comment
Share on other sites

[quote name='Olsberg' date='Aug 14 2008, 09:51 AM']This is a big improvement, and the possibility to search in plugin data is awesome!

One quite important thing is the handling of three-character words/abbrevations. Out of the box, it is currently not possible to search for that short words. I'd appreciate the possibility in IPB3. Many thanks!

You would need to either

1) Use sphinx
2) Edit your mysql configuration

The limitation is a MySQL limitation, not an IPB one specifically I'm afraid. By default (unless you edit your mysql config), MySQL only indexes 4 letter words and higher in it's fulltext indexes, which are used by IPB (and most other mysql search applications).

Link to comment
Share on other sites

My site is in Chinese UTF-8 encoding. The search function of IPB2 is not good, if I use Full Text Search to search Chinese words, the results are not as many as Manual Search. Also no keywords high lighted on both search types, unless I search english keywords. I think this problem exists not only in Chinese board, but also in Korean, Japanese, Thai, etc.

Hope in IPB3 this problem can be fixed.

Link to comment
Share on other sites

has to be the best improvement i've heard of so far, having wrote the search function for the ip.tracker based off ipb's i know how old it was and it was a drag having to do all that, now all i have to do is make a plugin, awesome!

Link to comment
Share on other sites

[quote name='Jaggi' date='Aug 14 2008, 09:58 AM']has to be the best improvement i've heard of so far, having wrote the search function for the ip.tracker based off ipb's i know how old it was and it was a drag having to do all that, now all i have to do is make a plugin, awesome!

yeah but after you already did the work on that now you will have to redo it also and learn a lot of new things about the IPB setup, which is going to be a pain, for me anyway.

Link to comment
Share on other sites

[quote name='BASHERS33' date='Aug 14 2008, 11:02 AM'][quote name='Jaggi' date='Aug 14 2008, 09:58 AM']has to be the best improvement i've heard of so far, having wrote the search function for the ip.tracker based off ipb's i know how old it was and it was a drag having to do all that, now all i have to do is make a plugin, awesome!

yeah but after you already did the work on that now you will have to redo it also and learn a lot of new things about the IPB setup, which is going to be a pain, for me anyway.


You don't have to control the searching at all, all you need todo is put your content into the search index, which is quite simple. You can add to the index with a call like: IPSSearchIndex::update( 'Your content here', 'Title of your content' ... ). There are a few more params, that tell the index where the content came from, but that's the gist of it :)

Link to comment
Share on other sites

[quote name='Josh' date='Aug 14 2008, 10:06 AM'][quote name='BASHERS33' date='Aug 14 2008, 11:02 AM'][quote name='Jaggi' date='Aug 14 2008, 09:58 AM']has to be the best improvement i've heard of so far, having wrote the search function for the ip.tracker based off ipb's i know how old it was and it was a drag having to do all that, now all i have to do is make a plugin, awesome!

yeah but after you already did the work on that now you will have to redo it also and learn a lot of new things about the IPB setup, which is going to be a pain, for me anyway.


You don't have to control the searching at all, all you need todo is put your content into the search index, which is quite simple. You can add to the index with a call like: IPSSearchIndex::update( 'Your content here', 'Title of your content' ... ). There are a few more params, that tell the index where the content came from, but that's the gist of it :)

ok, sounds simpler than I thought then. A lot of the changes announced thus far have sounded complicated, but maybe when I see them I will figure them out. Hopefully. :)

Link to comment
Share on other sites

[quote name='BASHERS33' date='Aug 14 2008, 04:33 PM'][quote name='Josh' date='Aug 14 2008, 10:06 AM'][quote name='BASHERS33' date='Aug 14 2008, 11:02 AM'][quote name='Jaggi' date='Aug 14 2008, 09:58 AM']has to be the best improvement i've heard of so far, having wrote the search function for the ip.tracker based off ipb's i know how old it was and it was a drag having to do all that, now all i have to do is make a plugin, awesome!

yeah but after you already did the work on that now you will have to redo it also and learn a lot of new things about the IPB setup, which is going to be a pain, for me anyway.


You don't have to control the searching at all, all you need todo is put your content into the search index, which is quite simple. You can add to the index with a call like: IPSSearchIndex::update( 'Your content here', 'Title of your content' ... ). There are a few more params, that tell the index where the content came from, but that's the gist of it :)

ok, sounds simpler than I thought then. A lot of the changes announced thus far have sounded complicated, but maybe when I see them I will figure them out. Hopefully. :)



dunno why your complaining bash i have to code it, lol.

I knew it wouldn't be that hard to do it aside from defining the db i guess and defining the fields to be indexed, thats probably about the gist of it.

Link to comment
Share on other sites

[quote name='TheMorrigan' date='Aug 14 2008, 01:18 PM']So its going to have a Live search similar to (pardon the gaming reference but its the first thing that comes to mind) Wowhead.com or Curse.com?

Yep, more or less just like wowhead.com :)

Link to comment
Share on other sites

[quote name='Josh' date='Aug 14 2008, 07:52 PM'][quote name='TheMorrigan' date='Aug 14 2008, 01:18 PM']So its going to have a Live search similar to (pardon the gaming reference but its the first thing that comes to mind) Wowhead.com or Curse.com?

Yep, more or less just like wowhead.com :)

WOW, this will rock!

Link to comment
Share on other sites

[quote name='bfarber' date='Aug 14 2008, 02:29 PM']The limitation is a MySQL limitation, not an IPB one specifically I'm afraid. By default (unless you edit your mysql config), MySQL only indexes 4 letter words and higher in it's fulltext indexes, which are used by IPB (and most other mysql search applications).
I'm using Sphinx, which works very well. My users can now perform two-letter searches, which is a big improvement over the default four-letter search. I run into this limitation time and time again when visiting Invision forums, so it's nice to see it being addressed..

The improved search functionality is a very welcome addition, and I appreciate the detail revealed here. :)

..Al

Link to comment
Share on other sites

[quote name='MRGTB' date='Aug 14 2008, 09:09 PM']very nice indeed, people have been asking for this to be done over at vBulletin, as I'm sure you know! :rolleyes:

People have been asking for us to do this since we first released Gallery and Blog, as I'm sure you know. ;) And that's been what....4 years? It's just such a major architectural change to how search works that it had to wait.

@AtariAge, I agree, Sphinx is great and it's nice that we will be supporting it out of the box. We use it on a number of our larger sites. I've thought about looking into other alternatives too (like FAST and Lucene). They probably won't make it into 3.0, but could likely debut in later releases as our search system is modular (hence the ease of the Sphinx integration in 2.x).

Link to comment
Share on other sites


I don't understand much of that is said, but does the above sentence mean spoilers will not be hidden when a user is searching?

To reduce the disk space overhead and make searches even quicker, the search index has a stripped down version of the content that contains no bbcode or markup of any kind.

Link to comment
Share on other sites




Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Add a comment...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...