Elasticsearch is slowing down the time to submit a post

[RevengeFNF]

I upgraded yesterday to 4.3.1 and i installed elasticsearch in an external server with a ping of only 5ms to my live server.

My users noticed that it was taking 1 or 2 seconds to submit every post they made. Then i changed back to mysql and they were correct that it was slower. Now its simple instant, i press submit reply and the post appears right away. 

My question is, how can Elasticsearch slowdown the post? The board only shows the post after it was also sent to Elasticsearch?

I'm going to install Elasticsearch on localhost to check if the problem its in the fact of me using an external server for elasticsearch.

[RevengeFNF]

After the test with Elasticsearch on localhost, submitting a post is fast like it was with mysql.

Latency and probably also less capable server, caused this.

My question here is, should not the post be submitted as fast as it can be, regardless if  Elasticsearch can keep up or not?

[Matt]

The search index is built right after the post is made.

To batch it would mean a pretty serious rewrite of the search index management.

A decent ES host should be very fast.

[RevengeFNF]

23 minutes ago, Matt said:

The search index is built right after the post is made.

To batch it would mean a pretty serious rewrite of the search index management.

A decent ES host should be very fast.

I'm going to stick with localhost since its fast and i have the resources in it.

The strange thing is that the Activity Streams were very fast and Searching was also fast. Only posting was not. Thanks for the clarification. 

[Charles]

We use a remote ES host here on this site without issue.

[RevengeFNF]

20 minutes ago, Charles said:

We use a remote ES host here on this site without issue.

That server is using a mechanical disk and the problem might be there, since i noticed Elasticsearch might be intensive in write tasks. My Post table is 1.4 Gb but the Elasticsearch Index is 6.5 Gb. Searching is fast because it is going to the Ram and not the Disk.

In localhost i use a SSD so that's its not an issue.

Since submitting a post is dependent on building the index in Elasticsearch, i can only suggest people to use localhost or a fast external server.

[DSystem]

3 hours ago, Matt said:

The search index is built right after the post is made.

I believe that the search index should be built in the background. This would avoid delaying the posts, regardless of the Elasticsearch server configuration.

[Joy Rex]

4 hours ago, RevengeFNF said:

I'm going to stick with localhost since its fast and i have the resources in it. 

Out of curiosity, what are your hosting specs for considering running ES on the same machine as IPS?

I'm getting ready to upgrade from 3.4.9 to 4.3.1, and I am excited by the prospect of ES, but I want to make sure I have the horsepower to accommodate it if possible and not have to deal with a 3rd party ES provider if at all possible.

[RevengeFNF]

6 minutes ago, Joy Rex said:

Out of curiosity, what are your hosting specs for considering running ES on the same machine as IPS?

I'm getting ready to upgrade from 3.4.9 to 4.3.1, and I am excited by the prospect of ES, but I want to make sure I have the horsepower to accommodate it if possible and not have to deal with a 3rd party ES provider if at all possible.

If your site is fast with mysql, it will also be fast with elasticsearch in localhost for sure. You just need to have some free Ram available, since elasticsearch likes it ;)

[Charles]

I would personally suggest using a professional hosted Elasticsearch service. There are many out there. They are low cost and then you don't have to think about anything. It will just work.

[USCJ Digital]

Is it possible to use Elasticsearch with a cloud-based community?

[RevengeFNF]

39 minutes ago, Charles said:

I would personally suggest using a professional hosted Elasticsearch service. There are many out there. They are low cost and then you don't have to think about anything. It will just work.

Its not that cheap. I went to Elasticsearch Cloud and the cheapest plan they have (it would work with my site), 1 Gb of Ram and 24 Gb of SSD Storage, would cost me 58$ per month.

[waccoe.com]

20 hours ago, RevengeFNF said:

Its not that cheap. I went to Elasticsearch Cloud and the cheapest plan they have (it would work with my site), 1 Gb of Ram and 24 Gb of SSD Storage, would cost me 58$ per month.

Yeah it's definitely not cheap.

external elastic will help only if you have resources issues that we don't have at all. Have reverted back to MySQL.

[Joy Rex]

On 5/4/2018 at 9:23 AM, RevengeFNF said:

If your site is fast with mysql, it will also be fast with elasticsearch in localhost for sure. You just need to have some free Ram available, since elasticsearch likes it ?

Thanks - what is the general recommendation for ES RAM? 1GB?

@Charles - while that makes sense, I think some IPS guidelines on best practices in implementing ES (hosted or otherwise) would go a long way on promoting a powerful feature IPS now has.

Considering too ES Cloud starts at 45USD/month, I don't think hosted is an attainable reality for a lot of forum owners. I know I would have a hard time essentially doubling my hosting costs just to support a search service.

[RevengeFNF]

54 minutes ago, Joy Rex said:

Thanks - what is the general recommendation for ES RAM? 1GB?

What is the size of your post table?

[Joy Rex]

23 hours ago, RevengeFNF said:

What is the size of your post table?

In terms of MB (this is on 3.4.9) my _posts table is 1404.38 MB.

Rows, it has 1826982.

[RevengeFNF]

56 minutes ago, Joy Rex said:

In terms of MB (this is on 3.4.9) my _posts table is 1404.38 MB.

Rows, it has 1826982.

You are in 3.4.9 and already thinking in ES for IPS 4.3? hehe

But from my testing, my post table is 1.4Gb and 1Gb dedicated to ES is enough.

[jair101]

I think in the previous discussion it was mentioned that the relevant table is core_search_index and not the posts. So basically, you should give the same RAM amount to elasticsearch as the current size of the search index. 

And yeah, Charles, it is a second time you recommend external service, but I believe you will be surprised how much difference there is between what you and what majority of IPS customers consider low cost :)

[RevengeFNF]

1 hour ago, jair101 said:

I think in the previous discussion it was mentioned that the relevant table is core_search_index and not the posts. So basically, you should give the same RAM amount to elasticsearch as the current size of the search index. 

The size of the core_search_index and the post table is almost the same.

That's not how Elasticsearch works. From my testing, 1 Gb is enough for a 1,4Gb table. It can even be less and im still testing.

Don't forget that putting more Ram dedicated to Elasticsearch than needed will hurt the performance, because of the way java garbage collection works.

[Chris027]

I have this long delay issue as well. I tried AWS elasticsearch but that wasn't as fast as mysql search. However, mysql search sucks. 

Now ES is running locally but the problem is really annoying. 

Anyone have any best practices for local ES to speed up things?

[maddog107_merged]

On 5/4/2018 at 2:41 AM, Matt said:

The search index is built right after the post is made.

To batch it would mean a pretty serious rewrite of the search index management.

A decent ES host should be very fast.

Matt, does that mean you are not using the "Bulk API" for elasticsearch?

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html

Trying to figure out why replies on huge threads (1000s of pages) take 20-30 seconds to go through.  

[Chris027]

Just now, maddog107_merged said:

Matt, does that mean you are not using the "Bulk API" for elasticsearch?

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html

Trying to figure out why replies on huge threads (1000s of pages) take 20-30 seconds to go through.  

+1

[yacenty]

we are 3rd day after enabling ElasticSearch (700k images, 150k topics, 2.5M post, 2.5M gallery comments) - the site gets slower!!! Especialy with saving new content.
Our elastic machine is 16CPU, 16GB ram, 50GB Enterprise SSD.

Guys, what are Your feelings about elastic in the IPB?

[Chris027]

38 minutes ago, yacenty said:

we are 3rd day after enabling ElasticSearch (700k images, 150k topics, 2.5M post, 2.5M gallery comments) - the site gets slower!!! Especialy with saving new content.
Our elastic machine is 16CPU, 16GB ram, 50GB Enterprise SSD.

Guys, what are Your feelings about elastic in the IPB?

It’s a trade off unfortunately. Search is way better but the slowness when saving new comments is a real pain for us. 

[maddog107_merged]

@yacenty @Chris027

 

Please see 

 

We have submitted a ticket and received a response from IPB today that they are working on a new/different fix. Hopefully it will resolve these issues.