Repeated errors running tasks

[willadamskeane]

Hey there,

I've been having trouble with the tasks set to run automatically on my board. This happens pretty much every time they run:

 

Looking at the support log, I see the following error occurring repeatedly in the logs:

Thanks for your help!

[Jim M]

Were you having any MySQL issues recently? After running through your queue, the rest of these tasks were able to run successfully without issue. If you continue to see these, please let us know.

[willadamskeane]

3 hours ago, Jim M said:

Were you having any MySQL issues recently? After running through your queue, the rest of these tasks were able to run successfully without issue. If you continue to see these, please let us know.

No SQL issues recently, as far as I know. The errors are continuing.

[Marc]

You would need to look further into this with your hosting company. The underlying error for this is indeed a mysql issue

IPS\Db\Exception::2002
Connection refused

[willadamskeane]

18 hours ago, Marc Stridgen said:

You would need to look further into this with your hosting company. The underlying error for this is indeed a mysql issue

IPS\Db\Exception::2002
Connection refused

I run self-managed dedicated servers, one for the frontend and one for the SQL database. I don't think the connection refused error is the root cause here, I think that's relatively uncommon compared to the frequency of the tasks failing. Did you get a chance to look at the other error messages appearing commonly in the logs?

[Marc]

I did indeed. Each of those comes up directly after a connection refused message.

To explain, the reason my colleague asked about the database, is that we see the error you mentioned, only when there is an error with the database itself. Essentially its trying to get information from the database and for whatever reason, its failing. Generally this tends to be crashed tables, limited connections, or indeed connection refused as you are getting in this case.

[willadamskeane]

On 9/8/2022 at 2:28 AM, Marc Stridgen said:

I did indeed. Each of those comes up directly after a connection refused message.

To explain, the reason my colleague asked about the database, is that we see the error you mentioned, only when there is an error with the database itself. Essentially its trying to get information from the database and for whatever reason, its failing. Generally this tends to be crashed tables, limited connections, or indeed connection refused as you are getting in this case.

I'm just trying to understand how I should go about resolving this issue. Are the warnings that appear in the dashboard safe to ignore, and if so is there any way to turn them off? I'm not sure why there would randomly be a connection refused error on the SQL server, as the forum otherwise runs fine and there haven't been any issues other than the occasional task that fails to run.

[Jim M]

57 minutes ago, willadamskeane said:

I'm just trying to understand how I should go about resolving this issue. Are the warnings that appear in the dashboard safe to ignore, and if so is there any way to turn them off? I'm not sure why there would randomly be a connection refused error on the SQL server, as the forum otherwise runs fine and there haven't been any issues other than the occasional task that fails to run.

They are not safe to ignore, it is an error connecting to your database which would need to be resolved. If these only come up with tasks, you may wish to ensure that there isn't some issue where CRON is running under a different PHP instance which is having some restriction. My suggestion would be to work with a server administrator, if you are unsure how to troubleshoot this.

[willadamskeane]

On 9/14/2022 at 12:18 PM, Jim M said:

They are not safe to ignore, it is an error connecting to your database which would need to be resolved. If these only come up with tasks, you may wish to ensure that there isn't some issue where CRON is running under a different PHP instance which is having some restriction. My suggestion would be to work with a server administrator, if you are unsure how to troubleshoot this.

It seems the MySQL "Connection refused" error is only being logged to the file-log, which is used when the database is inaccessible - makes sense. However, looking at these logs, the connection refused was last logged days ago, while I'm seeing very frequent errors running tasks, the last one happening today. If I look at the System logs in the Support section of the Admin CP, which are being logged to the database, I can see the error that is being reported - unrelated to the connection refused:

Quote

Error: Call to a member function language() on null (0)
#0 /var/www/html/system/Patterns/ActiveRecord.php(335): IPS\cms\_Databases->get__title()
#1 /var/www/html/applications/cms/Application.php(150): IPS\Patterns\_ActiveRecord->__get('_title')
#2 /var/www/html/system/Content/Item.php(1375): IPS\_Application::IPS\cms\{closure}('IPS\\cms\\Categor...')
#3 /var/www/html/applications/cms/sources/Records/Records.php(1822): IPS\Content\_Item->container()
#4 /var/www/html/system/Patterns/ActiveRecord.php(335): IPS\cms\_Records->get__forum_record()
#5 /var/www/html/applications/cms/sources/Records/Records.php(1681): IPS\Patterns\_ActiveRecord->__get('_forum_record')
#6 /var/www/html/applications/cms/sources/Records/Records.php(2180): IPS\cms\_Records->__get('_forum_record')
#7 /var/www/html/applications/cms/sources/Records/Records.php(231): IPS\cms\_Records->useForumComments()
#8 /var/www/html/system/Patterns/ActiveRecordIterator.php(76): IPS\cms\_Records::constructFromData(Array)
#9 /var/www/html/applications/core/extensions/core/Queue/MemberContent.php(93): IPS\Patterns\_ActiveRecordIterator->current()
#10 /var/www/html/system/Task/Task.php(47): IPS\core\extensions\core\Queue\_MemberContent->run(Array, 0)
#11 /var/www/html/applications/core/tasks/queue.php(43): IPS\_Task::runQueue()
#12 /var/www/html/system/Task/Task.php(367): IPS\core\tasks\_queue->IPS\core\tasks\{closure}()
#13 /var/www/html/applications/core/tasks/queue.php(55): IPS\_Task->runUntilTimeout(Object(Closure))
#14 /var/www/html/system/Task/Task.php(266): IPS\core\tasks\_queue->execute()
#15 /var/www/html/system/Task/Task.php(229): IPS\_Task->run()
#16 /var/www/html/applications/core/interface/task/task.php(58): IPS\_Task->runAndLog()
#17 {main}

This is the most frequent error, and it is occurring multiple times per day. It seems to clearly be a PHP execution error within the IP.Board application source files.

[Jim M]

Unfortunately, that also points to a database connection error when the language cannot be loaded from the database. 

[willadamskeane]

On 9/15/2022 at 4:19 PM, Jim M said:

Unfortunately, that also points to a database connection error when the language cannot be loaded from the database. 

Is there any way I can escalate this to an official support request? I have exhausted every resource I'm aware of in search of an answer.

[Jim M]

I'm afraid, I am not quite sure what you mean, this is an official support request and I am an official Invision support technician (hi! 🙂 ). 

My colleague and I have pointed out errors which are pointing to a server-related error with your MySQL server. Unfortunately, we only provide support for our software and any underlying server resources are outside our scope of support. If you are continuing to receive these errors, you may wish to hire a server administrator to aid your investigation.

[willadamskeane]

29 minutes ago, Jim M said:

I'm afraid, I am not quite sure what you mean, this is an official support request and I am an official Invision support technician (hi! 🙂 ). 

My colleague and I have pointed out errors which are pointing to a server-related error with your MySQL server. Unfortunately, we only provide support for our software and any underlying server resources are outside our scope of support. If you are continuing to receive these errors, you may wish to hire a server administrator to aid your investigation.

The problem I'm facing is that the application is reporting an error, which were it my own code I could fairly easily debug with inserted log statements or breakpoints. There are no errors being reported on the database server that would indicate a config issue, and MySQL errors can only be captured at the point of execution, usually in the application itself (in this case, IP.Board). Based on the errors being reported in the IP.Board system log, my best guess is that there is a database schema issue with a table, or a data integrity issue with one or more rows in a table, related to the CMS app (which I have a few custom databases setup in). I was just hoping that someone more familiar with that area of the code that the error is being reported in could point me the right direction.

[Jim M]

I have transferred this request to a ticket to see if there is any further information we can obtain for you. Please note there is no guarantee but as a courtesy, we can take a further look at this.