IPS Driver Error

[Kevin Leclair]

We are running version 3.0.5 and it seems at least every couple of weeks we get the IPS Driver Error problem on our website, http://seadoosportboats.com. I go in, repair the tables and everything seems to be fine.

Now my main issue with this is I have no way of knowing our website is physically down because of this error, unless I am viewing the website and see it for myself. At that time the website could of been down for hours, I really have no way of knowing how long it was actually down due to that error.

Is there a mod or can something be implemented so the board admins receive an email notification about this error specifically and maybe other errors so we can quickly remedy the issue? At this time there are no notifications around this error, so it requires someone constantly checking their website.

[Kevin Leclair]

Anyone?

[Ryan H.]

I would suggest filing a ticket and attempting to figure out what is causing the corrupted tables in the first place. It shouldn't happen.

[Kevin Leclair]

I understand it shouldn't happen, a lot of things shouldn't happen but they do. I am looking for a checks and balances system, even though the error shouldn't happen if it does then send me a notification so I know it did happen and I can resolve it immediately with minimal downtime...

[Connor T]

You could post the error from the driver error, and that would help determine more specifically. Some errors do send the administrator emails, however once the board goes down with driver error, I think that renders it all useless since usually the connection to the database dies.

[Wolfie]

So basically, you want it to cry out in pain/cry for help when it has something wrong in the database?

That would definitely be a nice option to have, especially if you can select which groups (that have ACP access) get the reports. Wouldn't matter if they have access to the SQL functions or not, because if there's a problem, then the first staff member to see the notice could always contact someone else who can do something about it.

Or perhaps somehow send it to the report center, so that when it's handled, it's know, instead of multiple admins doing a repair.

Either that, or someone could make an application. A task gets run periodically (however often you wish) doing a check on the database and when there's an error, it shows up to all admins. Then via the application, you can see the error it found along with a history of other errors it's found, who has done repairs in the past and other information. The task could also make a copy of the structure (not the data, just the structure), then note when it a change is detected and make a copy of the new structure. With that information, if things are running fine, then suddenly the structure is changed (application added) and problems start happening, you can figure out what may be causing the issue.

Obviously it would only make one copy of the structure and not make another copy until there is a change (ie, no duplicate copies). So keeping a history of the structure wouldn't take up that much space. Would be a very helpful tool.

[Connor T]

Almost all your ideas assume you have access to the database.

I'm saying in a board generalization that 60% of board errors are fatal involving the database, not allowing other functions to work. If anything, you could hard code an email or something into the driver error page, that only ONCE sends an email warning you of the time/error. Once the error log file is deleted, then another email can be sent.

[bfarber]

Wolfie, your ideas wouldn't work. It's assumed if the board is giving database errors serious enough to warrant immediate intervention that the database is (at least partially) inaccessible. We wouldn't be able to check groups (or settings in the database, for that matter), report center, etc.

Best we could do is allow you to add an email address to conf_global.php, and email that address if there's a db error. But that would mean EVERY db error would be emailed - it would be difficult to try to differentiate and only send if x error occurs, or if it occurs y number of times.

Not a bad idea, but didn't have much to add, which is why I didn't initially respond.

[Wolfie]

I was thinking more along the lines of the MySQL tool for checking the tables and if an error is reported there, then this would kick in. For severe database errors, obviously that may go from something simple like doing the repair to taking the board offline until it's fixed. The idea was more for errors that could be found and reported via the checking tool.

The idea is that if the problem escalates upon itself, then if it's caught and fixed as soon as it's noticed, it might prevent it from getting to a more severe point. Might even help debug why the problem is happening. Would be better off as an add-on though, because I doubt everyone would want such a tool running when they may have other measures in place.

[Kevin Leclair]

Brandon,

This is a step in the right direction, I like the conf_global.php idea because at least we get an email knowing that an error has occurred so we can promptly resolve it.

[Wolfie]

Brandon,

This is a step in the right direction, I like the conf_global.php idea because at least we get an email knowing that an error has occurred so we can promptly resolve it.

I think the problem he was trying to say with that is that you could potentially receive hundreds of emails before you notice the emails that are coming in. If your board gets (say) 100 visitors in an hour and if they each try to look at the board multiple times in that hour, hoping for the error to clear up, you can imagine the flood you'd get.

Also, even minor errors would end up getting emailed to you, which can sometimes happen for no reason at all and then be gone and never show again. Talk about a nightmare. :lol:

[Kevin Leclair]

In my opinion 100 emailed errors is better than not knowing of an error, so I am okay with that. If I start receiving a mass number of emails that just let's us know I need to resolve the error promptly.

[Robulosity2]

Or you know, you could not all be lazy and setup a Cron Job that emails you the days sql error log xx number of times a day.. Instead of asking for a feature that as Brandon & Terabyte commented is not overly fesable in the way you've all tried to ask for it.

Fun fact: If you have a need to be notified of when your site gets a driver error and have the ability to fix MySQL/SQL yourself.. than you should have the ability to setup an email cron to send you the same information.

[Kevin Leclair]

If you feel this a viable solution then post the syntax on how to setup the cron job so everyone interested can use it...

[Lindy]

Just as a general note.. if you're receiving that many database errors that frequently, you have larger issues at hand. Corrupt tables are almost always hosting related (as we often say, there's no "corrupt table" command that IP.Board executes.) You need to identify the root cause of the corruption, not just repair the table and move on over and over. It's usually caused by someone improperly shutting down MySQL, server being power cycled, or actual hardware issues.

There are many monitoring services and scripts (that you may run on your own server) that will monitor for specific content. If the software or service sees "IPS Driver Error" it can notify you. You may wish to investigate such a service - I'm not sure there's a terribly efficient yet reliable method for IP.Board to do it on its own.

[Kevin Leclair]

Thanks Lindy, can you please provide me the names of some of these services so I can investigate them?

[Robulosity2]

Webmin
SNMP
Nagios (Over kill)


For all intensive purposes, webmin is the easiest to use and won't have a large impact on your server (memory wise etc) and I've had no problems using it with even cPanel installed

SNMP is a little more technical, and you're best to have a monitoring client as well such as thedude

Re; cronjob thing

http://forums.fedoraforum.org/showthread.php?t=33614

[Kevin Leclair]

Re:

The cron job link -- that doesn't meet the needs of sending a notifier when the error happens, all that shows is how to email a log file. The site could of been down for hours by the time the log file is sent but thanks anyway.

[Wolfie]

You could always try to have something that will detect when the database is corrupt and have that run on a cron. One notification per (15/30/60min), could be done to email as well as to a phone.