eGullet Posted October 17, 2010 Share Posted October 17, 2010 This weekend we upgraded to 3.1.2 (which looks great, by the way). As part of the upgrade I generated a new sphinx.conf and installed it, then re-ran the indexer: all seemed well at that point. However, over the course of the day, our active content page gets messed up: you can see it here -- http://forums.egullet.org/index.php?app=core&module=search&do=active The first page looks OK: all the posts are in order, at any rate. But if you note the timestamp of the last post on that page, and flip over to page two, you start to see a problem: some of the posts on page two should have been on page one. And some are actually duplicates of posts that ARE shown on page one. If I run the complete Sphinx re-index this clears up temporarily, and slowly drifts back out of sync over the course of the day. Anyone have any idea how this could be happening? I didn't even realize that active content was using the Sphinx search. Maybe it's unrelated and the clearing up is coincidental (I only tried twice), but something odd is happening, regardless. Link to comment Share on other sites More sharing options...
thompsone Posted October 18, 2010 Share Posted October 18, 2010 That is odd. I suspect it's the way sphinx is indexing. What version of Sphinx are you running? Do you get any errors when (re)starting searchd? Did you build first and then rotate the indexes? Have you looked at the man pages here: Client Center > Documentation > Tutorials > Large Communities > ? Link to comment Share on other sites More sharing options...
eGullet Posted October 18, 2010 Author Share Posted October 18, 2010 This is Sphinx 0.9.9-release (r2117). The command I used was the conventional: /usr/local/bin/indexer --config /var/sphinx/sphinx.conf --all --rotate which seemed to work fine: a couple warnings as usual (duplicate document IDs in members_search and 'address' should be 'listen' Link to comment Share on other sites More sharing options...
thompsone Posted October 18, 2010 Share Posted October 18, 2010 Can you run a shell command? # service searchd restart Stopping searchd: [ OK ] Starting searchd: Sphinx 0.9.9-release (r2117) Copyright © 2001-2009, Andrew Aksyonoff using config file '/etc/sphinx/sphinx.conf'... listening on 127.0.0.1:9312 listening on all interfaces, port=3312 [ OK ] When the daemon restarts it will print errors/log to screen (shown above). You can post that here it might help. After the restart rotate your logs. I modified my sphinx.conf file to remove the errors and duplicates. I'll share it here so you can compare it to yours keeping in mind that our IPS applications may not be the same and also keep in mind that my sphinx installation may not be vanilla so my paths won't work for you or match yours. ############################################################################# ## data source definition ############################################################################# source ipb_source_config { #setup type = mysql sql_host = localhost sql_user = some_user sql_pass = some_password sql_db = some_database sql_port = 3306 } ############################### --- CORE --- ################################ source core_search_main : ipb_source_config { # Set our forum PID counter sql_query_pre = REPLACE INTO IVBcache_store VALUES( 'sphinx_core_counter', (SELECT max(id) FROM IVBfaq), '', 0, UNIX_TIMESTAMP() ) # Query posts for the main source sql_query = SELECT f.*, f.id as search_id, \ REPLACE( pi.perm_view, '*', 0 ) as perm_view \ FROM IVBfaq f \ LEFT JOIN IVBpermission_index pi ON ( pi.perm_type_id=1 AND pi.perm_type='help' ) \ WHERE f.id <= ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_core_counter' ) # Fields sql_attr_uint = search_id sql_ranged_throttle = 0 } source core_search_delta : core_search_main { # Override the base sql_query_pre sql_query_pre = # Query posts for the main source sql_query = SELECT f.*, f.id as search_id, \ REPLACE( pi.perm_view, '*', 0 ) as perm_view \ FROM IVBfaq f \ LEFT JOIN IVBpermission_index pi ON ( pi.perm_type_id=1 AND pi.perm_type='help' ) \ WHERE f.id > ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_core_counter' ) # Fields sql_attr_uint = search_id sql_ranged_throttle = 0 } index core_search_main { source = core_search_main path = /var/lib/sphinx/core_search_main docinfo = extern mlock = 0 morphology = none min_word_len = 2 charset_type = sbcs html_strip = 0 } index core_search_delta : core_search_main { source = core_search_delta path = /var/lib/sphinx/core_search_delta } ############################# --- CALENDAR --- ############################## source calendar_search_main : ipb_source_config { # Set our forum PID counter sql_query_pre = REPLACE INTO IVBcache_store VALUES( 'sphinx_calendar_counter', (SELECT max(event_id) FROM IVBcal_events), '', 0, UNIX_TIMESTAMP() ) # Query posts for the main source sql_query = SELECT e.*, e.event_id as search_id, \ REPLACE( pi.perm_view, '*', 0 ) as perm_view, \ CASE WHEN pi.authorized_users IS NULL THEN 0 ELSE pi.authorized_users END AS authorized_users, \ CASE WHEN pi.friend_only=0 THEN 0 ELSE e.event_member_id END AS friend_only, \ CASE WHEN pi.owner_only=0 THEN 0 ELSE e.event_member_id END AS owner_only \ FROM IVBcal_events e \ LEFT JOIN IVBpermission_index pi ON ( pi.perm_type_id=e.event_calendar_id AND pi.perm_type='calendar' ) \ WHERE e.event_id <= ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_calendar_counter' ) # Fields sql_attr_uint = search_id sql_attr_uint = friend_only sql_attr_uint = owner_only sql_attr_timestamp = event_unix_from sql_attr_multi = uint perm_view from field sql_attr_multi = uint authorized_users from field sql_attr_uint = event_member_id sql_ranged_throttle = 0 } source calendar_search_delta : calendar_search_main { # Override the base sql_query_pre sql_query_pre = # Query posts for the main source sql_query = SELECT e.*, e.event_id as search_id, \ REPLACE( pi.perm_view, '*', 0 ) as perm_view, \ CASE WHEN pi.authorized_users IS NULL THEN 0 ELSE pi.authorized_users END AS authorized_users, \ CASE WHEN pi.friend_only=0 THEN 0 ELSE e.event_member_id END AS friend_only, \ CASE WHEN pi.owner_only=0 THEN 0 ELSE e.event_member_id END AS owner_only \ FROM IVBcal_events e \ LEFT JOIN IVBpermission_index pi ON ( pi.perm_type_id=e.event_calendar_id AND pi.perm_type='calendar' ) \ WHERE e.event_id > ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_calendar_counter' ) } index calendar_search_main { source = calendar_search_main path = /var/lib/sphinx/calendar_search_main docinfo = extern mlock = 0 morphology = none min_word_len = 2 charset_type = sbcs html_strip = 0 } index calendar_search_delta : calendar_search_main { source = calendar_search_delta path = /var/lib/sphinx/calendar_search_delta } ################################# --- BLOG --- ############################## source blog_search_main : ipb_source_config { # Set our forum PID counter sql_query_pre = REPLACE INTO IVBcache_store VALUES( 'sphinx_blog_counter', (SELECT max(entry_id) FROM IVBblog_entries), '', 0, UNIX_TIMESTAMP() ) # Query posts for the main source sql_query = SELECT e.entry_id, e.entry_id as search_id, e.entry_name, e.entry_name as tordinal, e.entry, e.entry_date, e.entry_author_id, e.entry_num_comments, e.blog_id, \ b.blog_owner_only, b.blog_private, b.blog_disabled, \ CASE WHEN e.entry_status='published' THEN 0 ELSE e.entry_author_id END AS entry_not_published, \ CASE WHEN b.blog_owner_only=0 THEN 0 ELSE b.member_id END AS blog_owner_id, \ CASE WHEN b.blog_authorized_users IS NULL THEN 0 ELSE 1 END AS authorized_users \ FROM IVBblog_entries e \ LEFT JOIN IVBblog_blogs b ON ( b.blog_id=e.blog_id ) # Fields sql_attr_uint = search_id sql_attr_uint = blog_owner_only sql_attr_timestamp = entry_date sql_attr_str2ordinal = tordinal sql_attr_uint = entry_author_id sql_attr_uint = entry_num_comments sql_attr_uint = blog_id sql_attr_uint = blog_private sql_attr_uint = blog_disabled sql_attr_uint = entry_not_published sql_attr_uint = blog_owner_id sql_attr_uint = authorized_users sql_ranged_throttle = 0 } source blog_search_delta : blog_search_main { # Override the base sql_query_pre sql_query_pre = # Query posts for the delta source sql_query = SELECT e.entry_id, e.entry_id as search_id, e.entry_name, e.entry_name as tordinal, e.entry, e.entry_date, e.entry_author_id, e.entry_num_comments, e.blog_id, b.blog_owner_only, b.blog_private, b.blog_disabled, \ CASE WHEN e.entry_status='published' THEN 0 ELSE e.entry_author_id END AS entry_not_published, \ CASE WHEN b.blog_owner_only=0 THEN 0 ELSE b.member_id END as blog_owner_id, \ CASE WHEN b.blog_authorized_users IS NULL THEN 0 ELSE 1 END AS authorized_users \ FROM IVBblog_entries e \ LEFT JOIN IVBblog_blogs b ON ( b.blog_id=e.blog_id ) \ WHERE e.entry_id > ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_blog_counter' ) } index blog_search_main { source = blog_search_main path = /var/lib/sphinx/blog_search_main docinfo = extern mlock = 0 morphology = none min_word_len = 2 charset_type = sbcs html_strip = 0 #infix_fields = entry_name, entry #min_infix_len = 3 #enable_star = 1 } index blog_search_delta : blog_search_main { source = blog_search_delta path = /var/lib/sphinx/blog_search_delta } source blog_comments_main : ipb_source_config { # Set our forum PID counter sql_query_pre = REPLACE INTO IVBcache_store VALUES( 'sphinx_blog_comments_counter', (SELECT max(comment_id) FROM IVBblog_comments), '', 0, UNIX_TIMESTAMP() ) # Query posts for the main source sql_query = SELECT c.comment_id, c.comment_id as search_id, c.member_id as comment_member_id, c.comment_date, c.comment_queued, c.comment_text, \ e.entry_id, e.entry_name as tordinal, e.entry_date, e.entry_author_id, e.entry_num_comments, e.blog_id, \ b.blog_owner_only, b.blog_private, b.blog_disabled, \ CASE WHEN e.entry_status='published' THEN 0 ELSE e.entry_author_id END AS entry_not_published, \ CASE WHEN b.blog_owner_only=0 THEN 0 ELSE b.member_id END AS blog_owner_id, \ CASE WHEN b.blog_authorized_users IS NULL THEN 0 ELSE 1 END AS authorized_users, \ CONCAT(e.entry_last_comment_date, '.', e.entry_id ) as last_post_group \ FROM IVBblog_comments c \ LEFT JOIN IVBblog_entries e ON ( c.entry_id=e.entry_id ) \ LEFT JOIN IVBblog_blogs b ON ( b.blog_id=e.blog_id ) # Fields sql_attr_uint = search_id sql_attr_uint = entry_id sql_attr_uint = blog_owner_only sql_attr_timestamp = entry_date sql_attr_str2ordinal = tordinal sql_attr_uint = entry_author_id sql_attr_uint = entry_num_comments sql_attr_uint = blog_id sql_attr_uint = blog_private sql_attr_uint = blog_disabled sql_attr_uint = entry_not_published sql_attr_uint = blog_owner_id sql_attr_uint = authorized_users sql_attr_uint = last_post_group sql_attr_timestamp = comment_date sql_attr_uint = comment_member_id sql_attr_uint = comment_queued sql_ranged_throttle = 0 } source blog_comments_delta : blog_comments_main { # Override the base sql_query_pre sql_query_pre = # Query posts for the delta source sql_query = SELECT c.comment_id, c.comment_id as search_id, c.member_id as comment_member_id, c.comment_date, c.comment_queued, c.comment_text, \ e.entry_id, e.entry_name as tordinal, e.entry_date, e.entry_author_id, e.entry_num_comments, e.blog_id, \ b.blog_owner_only, b.blog_private, b.blog_disabled, \ CASE WHEN e.entry_status='published' THEN 0 ELSE e.entry_author_id END AS entry_not_published, \ CASE WHEN b.blog_owner_only=0 THEN 0 ELSE b.member_id END AS blog_owner_id, \ CASE WHEN b.blog_authorized_users IS NULL THEN 0 ELSE 1 END AS authorized_users, \ CONCAT(e.entry_last_comment_date, '.', e.entry_id ) as last_post_group \ FROM IVBblog_comments c \ LEFT JOIN IVBblog_entries e ON ( c.entry_id=e.entry_id ) \ LEFT JOIN IVBblog_blogs b ON ( b.blog_id=e.blog_id ) \ WHERE c.comment_id <= ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_blog_comments_counter' ) } index blog_comments_main { source = blog_comments_main path = /var/lib/sphinx/blog_comments_main docinfo = extern mlock = 0 morphology = none min_word_len = 2 charset_type = sbcs html_strip = 0 #infix_fields = comment_text #min_infix_len = 3 #enable_star = 1 } index blog_comments_delta : blog_comments_main { source = blog_comments_delta path = /var/lib/sphinx/blog_comments_delta } ############################## --- MEMBERS --- ############################## source members_search_main : ipb_source_config { # Set our forum PID counter sql_query_pre = REPLACE INTO IVBcache_store VALUES( 'sphinx_members_counter', (SELECT max(member_id) FROM IVBmembers), '', 0, UNIX_TIMESTAMP() ) # Query posts for the main source sql_query = SELECT m.member_id, m.member_id as search_id, m.member_group_id, m.email, m.joined, m.members_display_name, m.name, \ REPLACE( pi.perm_view, '*', 0 ) as perm_view, \ CASE WHEN pi.authorized_users IS NULL THEN 0 ELSE pi.authorized_users END AS authorized_users, \ CASE WHEN pi.friend_only=0 THEN 0 ELSE m.member_id END AS friend_only, \ CASE WHEN pi.owner_only=0 THEN 0 ELSE m.member_id END AS owner_only \ FROM IVBmembers m \ LEFT JOIN IVBpermission_index pi ON ( pi.perm_type_id=1 AND pi.perm_type='profile_view' ) \ WHERE m.member_id <= ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_members_counter' ) # Fields sql_attr_uint = search_id sql_attr_uint = friend_only sql_attr_uint = owner_only sql_attr_timestamp = joined sql_attr_multi = uint perm_view from field sql_attr_multi = uint authorized_users from field sql_ranged_throttle = 0 } source members_search_delta : members_search_main { # Override the base sql_query_pre sql_query_pre = # Query posts for the main source sql_query = SELECT m.member_id, m.member_id as search_id, m.member_group_id, m.email, m.joined, m.members_display_name, m.name, \ REPLACE( pi.perm_view, '*', 0 ) as perm_view, \ CASE WHEN pi.authorized_users IS NULL THEN 0 ELSE pi.authorized_users END AS authorized_users, \ CASE WHEN pi.friend_only=0 THEN 0 ELSE m.member_id END AS friend_only, \ CASE WHEN pi.owner_only=0 THEN 0 ELSE m.member_id END AS owner_only \ FROM IVBmembers m \ LEFT JOIN IVBpermission_index pi ON ( pi.perm_type_id=1 AND pi.perm_type='profile_view' ) \ WHERE m.member_id > ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_members_counter' ) } index members_search_main { source = members_search_main path = /var/lib/sphinx/members_search_main docinfo = extern mlock = 0 morphology = none min_word_len = 2 charset_type = sbcs html_strip = 0 #infix_fields = members_display_name, email, name #min_infix_len = 3 #enable_star = 1 } index members_search_delta : members_search_main { source = members_search_delta path = /var/lib/sphinx/members_search_delta } ############################## --- GALLERY --- ############################## source gallery_search_main : ipb_source_config { # Set our forum PID counter sql_query_pre = REPLACE INTO IVBcache_store VALUES( 'sphinx_gallery_counter', (SELECT max(id) FROM IVBgallery_images), '', 0, UNIX_TIMESTAMP() ) # Query posts for the main source sql_query = SELECT g.*, g.approved as image_approved, g.id as search_id, a.public_album \ FROM IVBgallery_images g \ LEFT JOIN IVBgallery_albums a ON ( g.album_id=a.id ) \ WHERE g.id <= ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_gallery_counter' ) # Fields sql_attr_uint = search_id sql_attr_timestamp = idate sql_attr_uint = member_id sql_attr_uint = album_id sql_attr_uint = category_id sql_attr_uint = views sql_attr_uint = comments sql_attr_uint = image_approved sql_ranged_throttle = 0 } source gallery_search_delta : gallery_search_main { # Override the base sql_query_pre sql_query_pre = # Query posts for the main source sql_query = SELECT g.*, g.approved as image_approved, g.id as search_id, a.public_album \ FROM IVBgallery_images g \ LEFT JOIN IVBgallery_albums a ON ( g.album_id=a.id ) \ WHERE g.id > ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_gallery_counter' ) } index gallery_search_main { source = gallery_search_main path = /var/lib/sphinx/gallery_search_main docinfo = extern mlock = 0 morphology = none min_word_len = 2 charset_type = sbcs html_strip = 0 } index gallery_search_delta : gallery_search_main { source = gallery_search_delta path = /var/lib/sphinx/gallery_search_delta } source gallery_search_comments : ipb_source_config { # Set our forum PID counter sql_query_pre = REPLACE INTO IVBcache_store VALUES( 'sphinx_IVBgallery_comments_counter', (SELECT max(pid) FROM IVBgallery_comments), '', 0, UNIX_TIMESTAMP() ) # Query posts for the main source sql_query = SELECT c.*, c.approved as comment_approved, c.pid as search_id, a.public_album, g.album_id, g.category_id \ FROM IVBgallery_comments c \ LEFT JOIN IVBgallery_images g ON ( g.id=c.img_id ) \ LEFT JOIN IVBgallery_albums a ON ( g.album_id=a.id ) \ WHERE c.pid <= ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_IVBgallery_comments_counter' ) # Fields sql_attr_uint = search_id sql_attr_timestamp = post_date sql_attr_uint = album_id sql_attr_uint = category_id sql_attr_uint = comment_approved sql_ranged_throttle = 0 } source gallery_search_comments_delta : gallery_search_comments { # Override the base sql_query_pre sql_query_pre = # Query posts for the main source sql_query = SELECT c.*, c.approved as comment_approved, c.pid as search_id, a.public_album, g.album_id, g.category_id \ FROM IVBgallery_comments c \ LEFT JOIN IVBgallery_images g ON ( g.id=c.img_id ) \ LEFT JOIN IVBgallery_albums a ON ( g.album_id=a.id ) \ WHERE c.pid > ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_IVBgallery_comments_counter' ) } index gallery_search_comments { source = gallery_search_comments path = /var/lib/sphinx/gallery_search_comments docinfo = extern mlock = 0 morphology = none min_word_len = 2 charset_type = sbcs html_strip = 0 } index gallery_search_comments_delta : gallery_search_comments { source = gallery_search_comments_delta path = /var/lib/sphinx/gallery_search_comments_delta } ################################# --- FORUM --- ############################## source forums_search_posts_main : ipb_source_config { # Set our forum PID counter sql_query_pre = REPLACE INTO IVBcache_store VALUES( 'sphinx_forums_counter_posts', (SELECT max(pid) FROM IVBposts), '', 0, UNIX_TIMESTAMP() ) # Query posts for the main source sql_query = SELECT p.pid, p.pid as search_id, p.author_id, p.post_date, p.post, p.topic_id, p.queued, \ t.tid, t.title, t.title as tordinal, t.views, t.posts, t.forum_id, t.last_post, t.state, t.start_date, t.starter_id, t.last_poster_id, t.topic_firstpost, \ CASE WHEN t.approved = -1 THEN 1 ELSE 0 END AS soft_deleted, \ CASE WHEN t.approved = -1 THEN 0 ELSE t.approved END AS approved, \ CONCAT(t.last_post, '.', t.tid ) as last_post_group \ FROM IVBposts p \ LEFT JOIN IVBtopics t ON ( p.topic_id=t.tid ) # Fields sql_attr_uint = queued sql_attr_uint = approved sql_attr_uint = soft_deleted sql_attr_uint = search_id sql_attr_uint = forum_id sql_attr_timestamp = post_date sql_attr_timestamp = last_post sql_attr_timestamp = start_date sql_attr_uint = author_id sql_attr_uint = starter_id sql_attr_uint = tid sql_attr_uint = posts sql_attr_uint = views sql_attr_str2ordinal = tordinal sql_attr_uint = last_post_group sql_ranged_throttle = 0 } source forums_search_posts_delta : forums_search_posts_main { # Override the base sql_query_pre sql_query_pre = # Query posts for the delta source sql_query = SELECT p.pid, p.pid as search_id, p.author_id, p.post_date, p.post, p.topic_id, p.queued, \ t.tid, t.title, t.title as tordinal, t.views, t.posts, t.forum_id, t.last_post, t.state, t.start_date, t.starter_id, t.last_poster_id, t.topic_firstpost, \ CASE WHEN t.approved = -1 THEN 1 ELSE 0 END AS soft_deleted, \ CASE WHEN t.approved = -1 THEN 0 ELSE t.approved END AS approved, \ CONCAT(t.last_post, '.', t.tid ) as last_post_group \ FROM IVBposts p \ LEFT JOIN IVBtopics t ON ( p.topic_id=t.tid ) \ WHERE p.pid > ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_forums_counter_posts' ) } index forums_search_posts_main { source = forums_search_posts_main path = /var/lib/sphinx/forums_search_posts_main docinfo = extern mlock = 0 morphology = none min_word_len = 2 charset_type = sbcs html_strip = 0 #infix_fields = post, title #min_infix_len = 3 #enable_star = 1 } index forums_search_posts_delta : forums_search_posts_main { source = forums_search_posts_delta path = /var/lib/sphinx/forums_search_posts_delta } ############################ --- DOWNLOADS --- ############################## source downloads_search_main : ipb_source_config { # Set our forum PID counter sql_query_pre = REPLACE INTO IVBcache_store VALUES( 'sphinx_downloads_counter', (SELECT max(file_id) FROM IVBdownloads_files), '', 0, UNIX_TIMESTAMP() ) # Query posts for the main source sql_query = SELECT f.*, f.file_id as search_id, \ REPLACE( pi.perm_view, '*', 0 ) as perm_view, \ CASE WHEN pi.authorized_users IS NULL THEN 0 ELSE pi.authorized_users END AS authorized_users, \ CASE WHEN pi.friend_only=0 THEN 0 ELSE f.file_submitter END AS friend_only, \ CASE WHEN pi.owner_only=0 THEN 0 ELSE f.file_submitter END AS owner_only \ FROM IVBdownloads_files f \ LEFT JOIN IVBpermission_index pi ON ( pi.perm_type_id=f.file_cat AND pi.app='downloads' AND pi.perm_type='cat' ) \ WHERE f.file_id <= ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_downloads_counter' ) # Fields sql_attr_uint = search_id sql_attr_uint = friend_only sql_attr_uint = file_cat sql_attr_uint = owner_only sql_attr_timestamp = file_updated sql_attr_multi = uint perm_view from field sql_attr_multi = uint authorized_users from field sql_attr_uint = file_submitter sql_ranged_throttle = 0 } source downloads_search_delta : downloads_search_main { # Override the base sql_query_pre sql_query_pre = # Query posts for the main source sql_query = SELECT f.*, f.file_id as search_id, \ REPLACE( pi.perm_view, '*', 0 ) as perm_view, \ CASE WHEN pi.authorized_users IS NULL THEN 0 ELSE pi.authorized_users END AS authorized_users, \ CASE WHEN pi.friend_only=0 THEN 0 ELSE f.file_submitter END AS friend_only, \ CASE WHEN pi.owner_only=0 THEN 0 ELSE f.file_submitter END AS owner_only \ FROM IVBdownloads_files f \ LEFT JOIN IVBpermission_index pi ON ( pi.perm_type_id=f.file_cat AND pi.app='downloads' AND pi.perm_type='cat' ) \ WHERE f.file_id > ( SELECT cs_value FROM IVBcache_store WHERE cs_key='sphinx_downloads_counter' ) } index downloads_search_main { source = downloads_search_main path = /var/lib/sphinx/downloads_search_main docinfo = extern mlock = 0 morphology = none min_word_len = 2 charset_type = sbcs html_strip = 0 } index downloads_search_delta : downloads_search_main { source = downloads_search_delta path = /var/lib/sphinx/downloads_search_delta } ############################################################################# ## indexer settings ############################################################################# indexer { mem_limit = 512M } ############################################################################# ## searchd settings ############################################################################# searchd { listen = 127.0.0.1 port = 3312 log = /var/lib/sphinx/log/searchd.log query_log = /var/lib/sphinx/log/query.log read_timeout = 5 max_children = 30 pid_file = /var/lib/sphinx/log/searchd.pid max_matches = 1000 seamless_rotate = 0 preopen_indexes = 0 unlink_old = 1 } # --eof-- Link to comment Share on other sites More sharing options...
eGullet Posted October 18, 2010 Author Share Posted October 18, 2010 Thanks for that post, it's always helpful to see someone else's configuration. To address a few of your points: I don't get any errors when I relaunch searchd, so that does not appear to be the issue. Second, our sphinx.conf files are identical for all practical purposes (I only checked the forums_search indices, but those are what matter for this problem). The only real explanations are that either Sphinx is failing to merge the results from the two indices together correctly, or the results are somehow being re-sorted in Invision's code (I don't think they do that, but you never know!). Can you take a look at your query.log file for an active topics search? Here is what I am seeing in mine for page 1: [Mon Oct 18 14:51:46.979 2010] 0.093 sec [scan/4/attr- 51 (0,25) @last_post_group] [forums_search_posts_main,forums_search_posts_delta] and for page 2: [Mon Oct 18 14:55:57.647 2010] 0.093 sec [scan/4/attr- 61 (25,25) @last_post_group] [forums_search_posts_main,forums_search_posts_delta] In particular, do you know if the order of the indices matter, and what the effect of the grouping is? Link to comment Share on other sites More sharing options...
thompsone Posted October 18, 2010 Share Posted October 18, 2010 I just examined our forums for topics and mine are sorting correctly so I'm guessing this has something to do with your installation. I just thought of something. Did you replace the sphinxapi.php file in your forum/site root with the one that came with the sphinx 9.9 distro or are you still using the old sphinxapi.php file? Okay back to your questions.Page 1 ------ [Mon Oct 18 16:34:00.170 2010] 0.025 sec [scan/4/attr- 34 (25,25) @last_post_group] [forums_search_posts_main,forums_search_posts_delta] Page 2 ------ [Mon Oct 18 16:34:01.502 2010] 0.025 sec [scan/4/attr- 0 (25,25) @last_post_group] [forums_search_posts_main,forums_search_posts_delta] Daily Cron ---------- Every 15 Min Cron ----------------- Can you tail your searchd.log and see if there are any clues there? [Mon Oct 18 16:30:01.399 2010] [13783] rotating indices (seamless=0) [Mon Oct 18 16:30:01.400 2010] [13783] rotating index 'core_search_delta': success [Mon Oct 18 16:30:01.401 2010] [13783] rotating index 'calendar_search_delta': success [Mon Oct 18 16:30:01.401 2010] [13783] rotating index 'blog_search_delta': success [Mon Oct 18 16:30:01.401 2010] [13783] rotating index 'members_search_delta': success [Mon Oct 18 16:30:01.402 2010] [13783] rotating index 'gallery_search_delta': success [Mon Oct 18 16:30:01.402 2010] [13783] rotating index 'gallery_search_comments_delta': success [Mon Oct 18 16:30:01.403 2010] [13783] rotating index 'forums_search_posts_delta': success [Mon Oct 18 16:30:01.403 2010] [13783] rotating index 'downloads_search_delta': success [Mon Oct 18 16:30:01.403 2010] [13783] rotating finished In your: /forum_directory/cache/ you will see both sphinx_error_numdate.cgi and sphinx_warning_numdate.cgi which are just log files they might yield some clues if the files are being generated. Link to comment Share on other sites More sharing options...
eGullet Posted October 19, 2010 Author Share Posted October 19, 2010 Our setups are practically identical: you are seeing exactly the same thing I am, except your works! I actually installed Sphinx a single time, back when we first upgraded to 3.0, and haven't touched the installation since. So I don't think the sphinxapi.php file should have changed. None of my log files show any sort of anomaly: I may have to get Invision hosting tech support on this, I think I'm out of my league here. Link to comment Share on other sites More sharing options...
thompsone Posted October 19, 2010 Share Posted October 19, 2010 Ok well it almost doesn't appear to be a sphinx issue then. In your ACP > Support > Check the file versions to make sure they are all 3.1.2 and your ftp client didn't upgrade or miss some of the files. Link to comment Share on other sites More sharing options...
eGullet Posted October 19, 2010 Author Share Posted October 19, 2010 The trouble is, what can it BE besides a Sphinx issue? I think Invision basically just dumps the search results to the screen, they don't really do anything to them beforehand. And the fact that rebuilding the indices clears it up? I've run every one of those little check scripts Invision has in the ACP, they all come up clean. Here's hoping someone there has some ideas... I really appreciate your help here: at least I don't appear to be missing anything completely stupid. Link to comment Share on other sites More sharing options...
thompsone Posted October 19, 2010 Share Posted October 19, 2010 In the ACP > Support > Diagnostics > Version Checker all the files to the right are in green and say 3.1.2? This looks more like a display issue and could even be in the skin, does this happen in the master skin? When I look at your new topics returns the last one on page 1 duplicates itself on page 2 at position 6. It's almost as if the template is requesting data in the old 3.0 format. So for me it boils down to 1) are the files all correct and 2) is the skin the issue. If sphinx wasn't working you'd get no returns at all or limited ones and that doesn't appear to be the issue. I'm sure they'll sort this out for you they have some really top notch people supporting the community. Good luck and let us know what you find. Link to comment Share on other sites More sharing options...
eGullet Posted October 19, 2010 Author Share Posted October 19, 2010 That's an interesting thought: the files that are still marked as 3.0.5 are files that don't exist in 3.1.2 (at least, not in my copy): among them are admin/applications/core/extensions/searchDisplay.php admin/applications/core/extensions/searchPlugin.php It seems to me that if they don't exist in the 3.1.2 upgrade then they shouldn't be getting called at all, but perhaps that is not the case. Do you have those files? ETA: I moved them out of the way and nothing changed, so they don't appear to have been the issue. And I get the exact same issue using the standard Invision skin as our custom skin, so it's not a skin issue (at least, not directly). Link to comment Share on other sites More sharing options...
thompsone Posted October 19, 2010 Share Posted October 19, 2010 I don't have any old files at all. I keep the file structure pretty tidy for housekeeping reasons. If your files are good, Sphinx is working, and the master skin shows the same problem there's not much left. The only other variables left are hardware, software related. I'm running a 64bit EL5 box with MySQL 5.1.50 and PHP 5.2.14 with Apache2 handler. Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.