IPS 4.3 - fallback to mysql if elasticsearch is down

[RevengeFNF]

Im testing the new IPS 4.3 beta, and i stopped elasticsearch on purpose to see what would happen, and View New Content and Search gives no results and i get this error in the logs: Failed to connect to xx.xx.xxx.xxx port 9200: Connection refused.

What i suggest is that if something bad happens to our elasticsearch server for some reason, IPS should fallback to Mysql, even if its only for the last 7 days, so View New Content can still work.

Edited March 22, 2018 by RevengeFNF

[Charles]

1 hour ago, RevengeFNF said:

Im testing the new IPS 4.3 beta, and i stopped elasticsearch on purpose to see what would happen, and View New Content and Search gives no results and i get this error in the logs: Failed to connect to xx.xx.xxx.xxx port 9200: Connection refused.

What i suggest is that if something bad happens to our elasticsearch server for some reason, IPS should fallback to Mysql, even if its only for the last 7 days, so View New Content can still work.

When you have Elasticsearch enabled we do not maintain the local MySQL search index to save on overhead.

[RevengeFNF]

8 minutes ago, Charles said:

When you have Elasticsearch enabled we do not maintain the local MySQL search index to save on overhead.

Imagine that we wake up just to find out that our elasticsearch server is down and we don't have access to it. What we will do is going to the Admin Panel and change the search to Mysql and rebuild the index.

What iam suggestion is for IPS to do that automatically in case the connection to elasticsearch fails, so our users don't lose access to the Activity Streams while we are away.

Offcourse im not expecting my elasticsearch server to fail. Its just a fallback that can be useful in case something bad happens.

Edited March 22, 2018 by RevengeFNF

[Optic14]

IMO your efforts would be much better spent ensuring you do have access to restart Elasticsearch (which only takes seconds if not minutes at the most). It would take much much longer to rebuild MySQL search index from scratch, and even if you do have it constantly rebuilt as a backup you would lose the huge space savings in your database by moving to Elasticsearch in the first place.

I do however agree though a more graceful error message should be given (e.g. Search temporarily unavailable" instead of returning 0 results, which could be frustrating)

Edited March 23, 2018 by Optic14

[Lindy]

If you have enough data to make Elasticsearch worthwhile, it stands to reason you can fix whatever the issue is with ES in the amount of time it's going to take to rebuild the search index locally. Either way, your users aren't going to have accurate activity streams immediately. I'd focus the effort on restoring service to your elasticsearch instance. 

[RevengeFNF]

4 hours ago, Lindy said:

If you have enough data to make Elasticsearch worthwhile, it stands to reason you can fix whatever the issue is with ES in the amount of time it's going to take to rebuild the search index locally. Either way, your users aren't going to have accurate activity streams immediately. I'd focus the effort on restoring service to your elasticsearch instance. 

Lindy, you know i understand about servers and im not expecting to have my server down, and if its down, i will put it up right away.

What im telling is that we are not online 24 hours a day. If elasticsearch fails, ips could fallback to mysql. It doesn't need to rebuild the whole index. IPS supports rebuilding only the last few days, that will take few space and its fast.

Like i said, just a suggestion and im not expecting my server to fail. ?

[loccom]

create a cron to restart ES if down.

[Darek_Hugo]

On 3/23/2018 at 6:53 AM, Lindy said:

If you have enough data to make Elasticsearch worthwhile, it stands to reason you can fix whatever the issue is with ES in the amount of time it's going to take to rebuild the search index locally. Either way, your users aren't going to have accurate activity streams immediately. I'd focus the effort on restoring service to your elasticsearch instance. 

Hello
How long will the data appear in the activity stream? I launched elasticsearch but users started reporting that no new content was appearing. Is there a solution for this?
Thank you for your help

[Morrigan]

@Hugo_S4 This isn't a tech support forum. You need to use Product Guidance or Peer to Peer help. However you need to see if its re-indexing your search results (would show you on the main Dashboard of the ACP). It starts from the oldest first so depending on the size of your community it can take a while.

If the re-index is not running you can run it from System > Search > Rebuild Search index in the top right.

[Zhana]

@Hugo_S4 I would suggest that you contact @ASTRAPI, he's the server expert.

[Darek_Hugo]

Just now, Morrigan

@Hugo_S4 This isn't a tech support forum. You need to use Product Guidance or Peer to Peer help. However you need to see if its re-indexing your search results (would show you on the main Dashboard of the ACP). It starts from the oldest first so depending on the size of your community it can take a while.

If the re-index is not running you can run it from System > Search > Rebuild Search index in the top right.

Increasing the amount of RAM available has solved the problem 😉