Jump to content

Community

IPS 4.3 - fallback to mysql if elasticsearch is down


Recommended Posts

Im testing the new IPS 4.3 beta, and i stopped elasticsearch on purpose to see what would happen, and View New Content and Search gives no results and i get this error in the logs: Failed to connect to xx.xx.xxx.xxx port 9200: Connection refused.

What i suggest is that if something bad happens to our elasticsearch server for some reason, IPS should fallback to Mysql, even if its only for the last 7 days, so View New Content can still work.

Edited by RevengeFNF
Link to post
Share on other sites
  • Management
1 hour ago, RevengeFNF said:

Im testing the new IPS 4.3 beta, and i stopped elasticsearch on purpose to see what would happen, and View New Content and Search gives no results and i get this error in the logs: Failed to connect to xx.xx.xxx.xxx port 9200: Connection refused.

What i suggest is that if something bad happens to our elasticsearch server for some reason, IPS should fallback to Mysql, even if its only for the last 7 days, so View New Content can still work.

When you have Elasticsearch enabled we do not maintain the local MySQL search index to save on overhead.

Link to post
Share on other sites
8 minutes ago, Charles said:

When you have Elasticsearch enabled we do not maintain the local MySQL search index to save on overhead.

Imagine that we wake up just to find out that our elasticsearch server is down and we don't have access to it. What we will do is going to the Admin Panel and change the search to Mysql and rebuild the index.

What iam suggestion is for IPS to do that automatically in case the connection to elasticsearch fails, so our users don't lose access to the Activity Streams while we are away.

Offcourse im not expecting my elasticsearch server to fail. Its just a fallback that can be useful in case something bad happens.

Edited by RevengeFNF
Link to post
Share on other sites

IMO your efforts would be much better spent ensuring you do have access to restart Elasticsearch (which only takes seconds if not minutes at the most). It would take much much longer to rebuild MySQL search index from scratch, and even if you do have it constantly rebuilt as a backup you would lose the huge space savings in your database by moving to Elasticsearch in the first place.

I do however agree though a more graceful error message should be given (e.g. Search temporarily unavailable" instead of returning 0 results, which could be frustrating)

Edited by Optic14
Link to post
Share on other sites
  • Management

If you have enough data to make Elasticsearch worthwhile, it stands to reason you can fix whatever the issue is with ES in the amount of time it's going to take to rebuild the search index locally. Either way, your users aren't going to have accurate activity streams immediately. I'd focus the effort on restoring service to your elasticsearch instance. 

Link to post
Share on other sites
4 hours ago, Lindy said:

If you have enough data to make Elasticsearch worthwhile, it stands to reason you can fix whatever the issue is with ES in the amount of time it's going to take to rebuild the search index locally. Either way, your users aren't going to have accurate activity streams immediately. I'd focus the effort on restoring service to your elasticsearch instance. 

Lindy, you know i understand about servers and im not expecting to have my server down, and if its down, i will put it up right away.

What im telling is that we are not online 24 hours a day. If elasticsearch fails, ips could fallback to mysql. It doesn't need to rebuild the whole index. IPS supports rebuilding only the last few days, that will take few space and its fast.

Like i said, just a suggestion and im not expecting my server to fail. ?

Link to post
Share on other sites
  • 1 year later...
On 3/23/2018 at 6:53 AM, Lindy said:

If you have enough data to make Elasticsearch worthwhile, it stands to reason you can fix whatever the issue is with ES in the amount of time it's going to take to rebuild the search index locally. Either way, your users aren't going to have accurate activity streams immediately. I'd focus the effort on restoring service to your elasticsearch instance. 

Hello
How long will the data appear in the activity stream? I launched elasticsearch but users started reporting that no new content was appearing. Is there a solution for this?
Thank you for your help

Link to post
Share on other sites

@Hugo_S4 This isn't a tech support forum. You need to use Product Guidance or Peer to Peer help. However you need to see if its re-indexing your search results (would show you on the main Dashboard of the ACP). It starts from the oldest first so depending on the size of your community it can take a while.

If the re-index is not running you can run it from System > Search > Rebuild Search index in the top right.

Link to post
Share on other sites
Just now, Morrigan

@Hugo_S4 This isn't a tech support forum. You need to use Product Guidance or Peer to Peer help. However you need to see if its re-indexing your search results (would show you on the main Dashboard of the ACP). It starts from the oldest first so depending on the size of your community it can take a while.

If the re-index is not running you can run it from System > Search > Rebuild Search index in the top right.

Increasing the amount of RAM available has solved the problem 😉

Link to post
Share on other sites
Guest
This topic is now closed to further replies.
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...

Important Information

We use technologies, such as cookies, to customise content and advertising, to provide social media features and to analyse traffic to the site. We also share information about your use of our site with our trusted social media, advertising and analytics partners. See more about cookies and our Privacy Policy