URL Searches - Timeouts

[Como]

Hi,

When I try searching for a domain, the search goes on for several minutes, eventually timing out and returning a 504 error. I have the system defaulted to 'all terms, but otherwise uses default settings.

This has been going on for at least two days. It is possible that it is connected with the latest update.

[Marc]

Could you give me an example search you are trying on your community? I can then get this looked at for you. Assuming this is just in the main ACP search, is that correct?

[Como]

11 minutes ago, Marc said:

Could you give me an example search you are trying on your community? I can then get this looked at for you. Assuming this is just in the main ACP search, is that correct?

It appears that all search is affected. Certainly the main search times out. As does the search box from the home page. I just tested with a thread, and it returned the searched for link. And it also worked at the forum level. (But I am uncertain about the very largest forums.)

I've also noticed that it does not work for hyperlinks or long links, at any level - only shorter (non-querying?), plain links.

I've tried a number of links - they all fail (time out - 504) at the site level.

[Marc]

This is only when you search links specifically? Could you send me some examples, as Im failing to replicate

Feel free to PM them

 

[Como]

Thanks, @Marc. I've sent you a message.

[Marc]

I've created a ticket on this for you, so we can look into it further

[Como]

Thanks, @Marc. I've received your email.

[Como]

Hi,

Just to let you know, that during the search index rebuild, links were returned. When I checked again after the index was completed, they again failed.

Although a bit slow, returns for all other searches are reliable, including those using the AND operator (which I expect would be more CPU intensive). It was suggested to me by an Invision tech that the size of the database might be the cause, but why are only URLs negatively affected?

[Jim M]

4 hours ago, Como said:

It was suggested to me by an Invision tech that the size of the database might be the cause, but why are only URLs negatively affected?

As you have such a large database any intense operations, like large text comparisons, would take longer and possibly timeout. As the developer mentioned, MySQL search isn't that performant with as many records as you have so the options are to limit the content which is contained in the search index or switch to ElasticSearch.

[Como]

Hi @Jim M

Thank you for the reply.

I did have a look at Elastic Search - it is very difficult to estimate the costs. But even the minimum fee is quite high. I was not really expecting to pay a 'how long is a bit of string' fee for something which seems like a core function.

Can anyone here tell what kind of plan I would likely require from Elastic, indexing 3.5 million posts?

Frankly, I have no idea how to the estimate costs. If anyone can clue me in, I'm all ears.

https://www.elastic.co/pricing

[Como]

Hi again @Jim M

I few tests seem to indicate that it is not just URLs, but any string containing hyphens or periods.

If I search only titles, although a bit slow, phases like 'geo-location' are returned. If, however, I search titles and content, it times out.

If I remove hyphens and split up URLs, this also works. As does keeping the hyphens and adding a space after them.

It seems that any string of characters separated by hyphens and/or periods are either not indexed, or fails the search (I am unsure which).

Test (plain text - not URLs):

one/two/three/four/five/six/seven/eight/nine/ten (successful)

one_two/three/four/five/six/seven/eight/nine/ten (successful)

one.two/three/four/five/six/seven/eight/nine/ten (unsuccessful)

one/two/three/four/five/six/seven/eight/nine.ten (unsuccessful)

one-two/three/four/five/six/seven/eight/nine/ten (unsuccessful)

This has something to do with how text containing certain characters is indexed and/or failure in their retrieval.