SebastienG Posted May 4, 2018 Posted May 4, 2018 Hello I have enabled elasticsearch on my board. Everything is working fine except the view updates task. When the tache is executed. 4C124/6 The task you attempted to run could not be found If i disable the elastic update on the viewupdate.php file, everything is working fine. If i disable the use of elasticsearch, everithing is working fine. static::$instance = new \IPS\Content\Search\Elastic\Index( \IPS\Http\Url::external( \IPS\Settings::i()->search_elastic_server . '/' . \IPS\Settings::i()->search_elastic_index ) ); I think it's a timeout problem. If the task is executed every second, their no problem. I have some IPS\Http\Request\CurlException (28) Operation timed out after 10001 milliseconds with 0 bytes received on my log and i recently increased index.max_result_window at 50000 after some errors
waccoe.com Posted May 5, 2018 Posted May 5, 2018 22 hours ago, SebastienG said: Hello I have enabled elasticsearch on my board. Everything is working fine except the view updates task. When the tache is executed. 4C124/6 The task you attempted to run could not be found If i disable the elastic update on the viewupdate.php file, everything is working fine. If i disable the use of elasticsearch, everithing is working fine. static::$instance = new \IPS\Content\Search\Elastic\Index( \IPS\Http\Url::external( \IPS\Settings::i()->search_elastic_server . '/' . \IPS\Settings::i()->search_elastic_index ) ); I think it's a timeout problem. If the task is executed every second, their no problem. I have some IPS\Http\Request\CurlException (28) Operation timed out after 10001 milliseconds with 0 bytes received on my log and i recently increased index.max_result_window at 50000 after some errors We are also experiencing the same issue. Support have said it is a server side problem, and our host said it is a problem with the elasticsearch, so we have left the search as MySQL Quote log error: Result window is too large, from + size must be less than or equal to: [10000] but was [2039925] We can increase the result window from default 10.000 up to 3.000.000 but that is not recommended at all from elastic as that will consume a lot of memory and possibly crash elastic. external elastic will help only if you have resources issues that we don't have at all
ASTRAPI Posted May 5, 2018 Posted May 5, 2018 I can see also the same timeout issue and on the log file of elastic i can see the result window warning.... On the log file there is a note also for some deprecated functions used but i think IPB developers will use newer methods soon so they will be compatible with latest versions and avoid any upgrade issues that elastic will remove the old functions..... ? All the rest are working great ! The issue is only with the view updates.... I did also a re installation of elastic to be sure that there is no other issues (configuration/performance e.t.c) but the issue exists ?
Linguica Posted May 16, 2018 Posted May 16, 2018 I noticed after updating to 4.3 that topic view counters were not updating. I do not have the new redis caching enable but I do have Elasticsearch configured. After some hunting, I discovered what I believe to be the culprit: viewupdates.php grabs the first 20 additional view counts from the core_view_updates database table it attempts to update the view count for each of them in update(), it actually updates the view count of the first content item using the count it calculated It then attempts to run \IPS\Content\Search\Index::i()->updateViewCounts and, in my case, times out, because there is some problem (not sure if the view count it is trying to update is missing from the elasticsearch cache, or what, but that's not the first order problem here anyway) It then throws an exception, but crucially, it never actually deletes the records from core_view_updates that it just updated the count with This leads to two separate problems: View counts for content items are not updated and the number of records in core_view_updates grows bigger (I had around 50,000 records to process by the time I realized what was going on) The view count for whatever item happens to be the first one that the task attempts to update gets constantly inflated, and at an increasing rate as time goes on, because every 5 minute the task increases its view count by however many views it has accumulated since the problem began before throwing an exception and erroring out
Chris027 Posted May 24, 2018 Posted May 24, 2018 I have this same problem. Here's what cron send me upon the error: Exception:IPS\Http\Request\CurlException Object( [message:protected] => Operation timed out after 10002 milliseconds with 0 bytes received [string:Exception:private] => [code:protected] => 28 [file:protected] => /srv/http/computeraudiophile.com/root/html/system/Http/Request/Curl.php [line:protected] => 383 [trace:Exception:private] => Array ( [0] => Array ( [file] => /srv/http/computeraudiophile.com/root/html/system/Http/Request/Curl.php [line] => 407 [function] => _execute [class] => IPS\Http\Request\_Curl [type] => -> [args] => Array ( ) ) [1] => Array ( [file] => /srv/http/computeraudiophile.com/root/html/system/Http/Request/Curl.php [line] => 302 [function] => _executeAndFollowRedirects [class] => IPS\Http\Request\_Curl [type] => -> [args] => Array ( [0] => POST [1] => {"script":{"inline":"ctx._source.index_views += 1;","lang":"painless"},"query":{"bool":{"must":[{"terms":{"index_class":["IPS\\forums\\Topic","IPS\\forums\\Topic\\Post"]}},{"term":{"index_item_id":7577}}]}}} ) ) [2] => Array ( [file] => /srv/http/computeraudiophile.com/root/html/system/Content/Search/Elastic/Index.php [line] => 227 [function] => post [class] => IPS\Http\Request\_Curl [type] => -> [args] => Array ( [0] => {"script":{"inline":"ctx._source.index_views += 1;","lang":"painless"},"query":{"bool":{"must":[{"terms":{"index_class":["IPS\\forums\\Topic","IPS\\forums\\Topic\\Post"]}},{"term":{"index_item_id":7577}}]}}} ) ) [3] => Array ( [file] => /srv/http/computeraudiophile.com/root/html/applications/core/tasks/viewupdates.php [line] => 116 [function] => updateViewCounts [class] => IPS\Content\Search\Elastic\_Index [type] => -> [args] => Array ( [0] => IPS\forums\Topic [1] => 7577 [2] => 1 ) ) [4] => Array ( [file] => /srv/http/computeraudiophile.com/root/html/applications/core/tasks/viewupdates.php [line] => 49 [function] => update [class] => IPS\core\tasks\_viewupdates [type] => -> [args] => Array ( [0] => IPS\forums\Topic [1] => 7577 [2] => 1 ) ) [5] => Array ( [function] => IPS\core\tasks\{closure} [class] => IPS\core\tasks\_viewupdates [type] => -> [args] => Array ( ) ) [6] => Array ( [file] => /srv/http/computeraudiophile.com/root/html/system/Task/Task.php [line] => 347 [function] => call_user_func [args] => Array ( [0] => Closure Object ( [this] => IPS\core\tasks\viewupdates Object ( [_data:protected] => Array ( [id] => 827 [app] => core [key] => viewupdates [frequency] => P0Y0M0DT0H5M0S [next_run] => 1527186841 [running] => 1 [enabled] => 1 [plugin] => [lock_count] => 0 [last_run] => 1527186576 ) [_new:protected] => [changed] => Array ( ) [skipCloneDuplication] => ) ) ) ) [7] => Array ( [file] => /srv/http/computeraudiophile.com/root/html/applications/core/tasks/viewupdates.php [line] => 93 [function] => runUntilTimeout [class] => IPS\_Task [type] => -> [args] => Array ( [0] => Closure Object ( [this] => IPS\core\tasks\viewupdates Object ( [_data:protected] => Array ( [id] => 827 [app] => core [key] => viewupdates [frequency] => P0Y0M0DT0H5M0S [next_run] => 1527186841 [running] => 1 [enabled] => 1 [plugin] => [lock_count] => 0 [last_run] => 1527186576 ) [_new:protected] => [changed] => Array ( ) [skipCloneDuplication] => ) ) ) ) [8] => Array ( [file] => /srv/http/computeraudiophile.com/root/html/system/Task/Task.php [line] => 248 [function] => execute [class] => IPS\core\tasks\_viewupdates [type] => -> [args] => Array ( ) ) [9] => Array ( [file] => /srv/http/computeraudiophile.com/root/html/system/Task/Task.php [line] => 217 [function] => run [class] => IPS\_Task [type] => -> [args] => Array ( ) ) [10] => Array ( [file] => /srv/http/computeraudiophile.com/root/html/applications/core/interface/task/task.php [line] => 62 [function] => runAndLog [class] => IPS\_Task [type] => -> [args] => Array ( ) ) ) [previous:Exception:private] => )
Chris027 Posted May 24, 2018 Posted May 24, 2018 Hi Guys - We had cURL version 7.29 installed and I believe everything was working. We updated to cURL version 7.6 today and I started noticing these errors. What version of cURL is everyone else using
Management Lindy Posted May 31, 2018 Management Posted May 31, 2018 For those having this issue, a patch will be released shortly and available by running the support wizard in the ACP. It should take care of the timeout issue.
Daniel F Posted May 31, 2018 Posted May 31, 2018 The patch is available:) Please run the support tools to apply it
maddog107_merged Posted May 31, 2018 Posted May 31, 2018 23 minutes ago, Daniel F said: The patch is available:) Please run the support tools to apply it Is the download from the client area updated? We cant use the wizard as we dont use FTP (please make it use SSH/keys).
TSP Posted May 31, 2018 Posted May 31, 2018 30 minutes ago, maddog107_merged said: Is the download from the client area updated? We cant use the wizard as we dont use FTP (please make it use SSH/keys). Yes it is.
maddog107_merged Posted May 31, 2018 Posted May 31, 2018 6 minutes ago, TSP said: Yes it is. It *appears* to have resolved the issue. I dont see the message in my dashboard anymore. Thanks ?
Linguica Posted May 31, 2018 Posted May 31, 2018 From looking at a diff it appears that this patch raises the timeout for a curl request in the viewupdates task from 10 seconds to 30 seconds.
Daniel F Posted May 31, 2018 Posted May 31, 2018 37 minutes ago, Linguica said: From looking at a diff it appears that this patch raises the timeout for a curl request in the viewupdates task from 10 seconds to 30 seconds. Exactly, we're using now the \IPS\LONG_REQUEST_TIMEOUT constant (which has indeed 30 seconds as default value) for the request.
maddog107_merged Posted May 31, 2018 Posted May 31, 2018 20 minutes ago, Daniel F said: Exactly, we're using now the \IPS\LONG_REQUEST_TIMEOUT constant (which has indeed 30 seconds as default value) for the request. Looks like the issue is back for me ? Maybe it was a problem on my side - lets give it a few hours to see if its fixed. Sorry about that.
maddog107_merged Posted May 31, 2018 Posted May 31, 2018 @Daniel F @Lindy Yup still broken for us ? 30secs is not enough I guess.
Linguica Posted May 31, 2018 Posted May 31, 2018 My understanding is that the basic way the elasticsearch engine works is that it stores a record for every single content item, including every single post in every topic, and when it runs the task it asks updateViewCounts() to update the "index_views" for every single post within a topic at once, so if you have exceptionally long topics, it can be trying to update hundreds or thousands of records at once. The simple answer is to just not have really long topics, of course, but that's not a very elegant solution.
maddog107_merged Posted May 31, 2018 Posted May 31, 2018 13 minutes ago, Linguica said: My understanding is that the basic way the elasticsearch engine works is that it stores a record for every single content item, including every single post in every topic, and when it runs the task it asks updateViewCounts() to update the "index_views" for every single post within a topic at once, so if you have exceptionally long topics, it can be trying to update hundreds or thousands of records at once. The simple answer is to just not have really long topics, of course, but that's not a very elegant solution. We have many threads that are thousands of pages long. Why would it store the same value with every single post though? Woudlnt it be more efficient to simply store it with the topic itself, you dont keep track of post views, just topic views?
Linguica Posted June 2, 2018 Posted June 2, 2018 I think the disconnect is that IPS is treating elasticsearch as a service that you pay someone else to make fast for you: On 1/4/2018 at 3:43 AM, Charles said: I personally never bothered learning to install Elasticsearch and neither did our dev team. There are dozens of places out there that offer hosted Elasticsearch. I used AWS and with a few clicks had it up and running. It's 2018: installing stuff on a VPS manually is soooo 2003. At the very least, IPS should make it clear that self-hosted elasticsearch is unsupported, if this is the case.
bfarber Posted June 4, 2018 Posted June 4, 2018 Much like anything self-hosted, we cannot take responsibility for installing or configuring third party software.
Chris027 Posted June 4, 2018 Posted June 4, 2018 I've asked for recommendations as well and received nothing. At least you guys could provide some recommendations for a hosted solution. I went to Elastic.co and found its hosted solution but it requires a little code change, not simply entering the URL and port in IPS. Sure I can do this, but hey wouldn't it be cool if we actually had recommendations for services that work out of the box. Doesn't seem too hard.
maddog107_merged Posted June 8, 2018 Posted June 8, 2018 On 5/31/2018 at 3:16 PM, Linguica said: My understanding is that the basic way the elasticsearch engine works is that it stores a record for every single content item, including every single post in every topic, and when it runs the task it asks updateViewCounts() to update the "index_views" for every single post within a topic at once, so if you have exceptionally long topics, it can be trying to update hundreds or thousands of records at once. The simple answer is to just not have really long topics, of course, but that's not a very elegant solution. @bfarber can you confirm/deny what Linguica mentioned above? I have threads with thousands of pages, could that be the source of the issue? @Real Hal9000 do you also have large amount of pages per thread?
Real Hal9000 Posted June 9, 2018 Posted June 9, 2018 19 hours ago, maddog107_merged said: @bfarber can you confirm/deny what Linguica mentioned above? I have threads with thousands of pages, could that be the source of the issue? @Real Hal9000 do you also have large amount of pages per thread? We locked some topics with way too many replies (>12.000) where each reply could take even 10-15 sec to be posted. After installing the latest patch, timeouts disappeared.
kalin Posted June 11, 2018 Posted June 11, 2018 On 6/9/2018 at 9:56 PM, Real Hal9000 said: We locked some topics with way too many replies (>12.000) where each reply could take even 10-15 sec to be posted. After installing the latest patch, timeouts disappeared. Same here
ExiledVip3r Posted June 12, 2018 Posted June 12, 2018 The patch mentioned above didn't fix it for me. View counts on all topics still wouldn't update as long as elasticsearch was being used. Ended up having to go back to MySQL search to get topic views updating again. Which sucks seeing as bad search results was one of my biggest complaints from users with 4.2 and older. Tried both using and not using the new Redis method for storing views, neither way helped. Have a handful of topics (20 or 30ish) that are >5000 replies if that's what matters here. 2.1 GB core_search_index table when using MySQL as well.
Recommended Posts
Archived
This topic is now archived and is closed to further replies.