Jump to content

Problem with `core_stream_subscriptions` causing huge increase in load


Recommended Posts

Just thought you guys should know that there are instances where the cron job can go crazy and apparently get into an infinite loop. It does a lot of this query…

/*bzone::bzone::IPS\core\Stream\_Subscription::sendBatch:40*/ SELECT DISTINCT *, core_members.last_visit FROM `core_stream_subscriptions`  LEFT JOIN `core_members` ON core_stream_subscriptions.member_id=core_members.member_id WHERE frequency = ? AND sent < ? and last_visit > ? ORDER BY sent ASC LIMIT 0,50

Backing up a bit, a couple weeks ago I noticed that load and CPU usage on my server doubled suddenly around January 16th. One of the symptoms was that there were frequently 2 php threads using 50% CPU each (and MySQL was using far more CPU than normal as well). Based on what my host told me I knew they were cron jobs for IP.Board. When I'd watch the queries that were running in MySQL everything seemed pretty normal. Anyway, finally I took a closer look at the queries that were being called and realized that query above shouldn't be called so frequently. I checked and I only had 5 `core_stream_subscriptions` records. Then I noticed two of them had `sent` values that were old. One of them corresponded with the date when the problem started, so I deleted those two records and my server went back to normal. Unfortunately I didn't delve further before deleting the records, so I'm not sure what exactly about those records were causing the problem.

Anyway, you might want to look into situations that can cause infinite loops that constantly call the SQL above.

Edited by jay5r
Link to comment
Share on other sites

Unfortunately, as you have removed the given issue here, it will be a little hard for us to investigate what is happening here without any indication to the stream or record which was causing issues. It is worth mentioning that in 4.6.10 there were some bug fixes to stream and a bug patch to resolve an issue. If this does come up again, if you don't mind grabbing the settings of the stream before deleting it that my be enough for us to investigate. Even better though would be to setup a test environment for us to take a look. Thanks.

Link to comment
Share on other sites

12 hours ago, jay5r said:

I can say that 4.6.10 didn't fix the problem. It started before I did the upgrade and didn't go away with the upgrade.

I just looked at my backup and I do have the data. I could restore it pretty easily.

If its corrupt data that was causing the problem, its likely  that data was created by an old version somewhere. I Would suggest just letting us know if you see the issue again, rather than restoring what you already know to be a problem

Link to comment
Share on other sites

9 hours ago, Marc Stridgen said:

If its corrupt data that was causing the problem, its likely  that data was created by an old version somewhere. I Would suggest just letting us know if you see the issue again, rather than restoring what you already know to be a problem

Define "corrupt". This would be all I'd have to do to restore the data…

INSERT INTO `core_stream_subscriptions` (`id`, `member_id`, `stream_id`, `frequency`, `sent`, `added`) VALUES (2,99547,1,'daily',1642337822,1636000819),(8,101722,4,'daily',1642265420,1642351820);

I've verified that the members and the streams exist.

Link to comment
Share on other sites

15 hours ago, jay5r said:

Define "corrupt". This would be all I'd have to do to restore the data…

INSERT INTO `core_stream_subscriptions` (`id`, `member_id`, `stream_id`, `frequency`, `sent`, `added`) VALUES (2,99547,1,'daily',1642337822,1636000819),(8,101722,4,'daily',1642265420,1642351820);

I've verified that the members and the streams exist.

As that was generated by an old version however, I Would recommend letting us know if you see the problem happen again naturally, rather than inserting that set of data manually. While I understand what you are saying there, clearly there is something off with that specific item

Link to comment
Share on other sites

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...