Jump to content

Saving posts is slow


James25
 Share

Recommended Posts

What can be done about saving posts being slow? There are a large number of people following some threads, and I'm thinking that may be related.

What can be done to speed it up? I have IPB set to use cron, if that matters.

Thanks!

James

Link to comment
Share on other sites

How many is a "large number"? If there are 50 or less (remembering off the top of my head here) we send the notifications immediately, but if there are more we queue the notifications to be sent by a background task. If we are sending notifications immediately, the speed of your email server will make a difference, but if we queue them then emailing is not likely going to be the problem.

Do you use Elasticsearch? It's possible this is slowing down posting if the newly posted content has to be pushed across the internet to a slow ES instance.

Any custom applications or plugins? Many are designed to do X when Y happens, which often includes submitting a new post or topic.

Link to comment
Share on other sites

48 minutes ago, bfarber said:

How many is a "large number"? If there are 50 or less (remembering off the top of my head here) we send the notifications immediately, but if there are more we queue the notifications to be sent by a background task. If we are sending notifications immediately, the speed of your email server will make a difference, but if we queue them then emailing is not likely going to be the problem.

Is this 50 number configurable? I like the idea of queuing all e-mails as an option. I think sendgrid sometimes introduces a bit of lag.

Link to comment
Share on other sites

29 minutes ago, CoffeeCake said:

Is this 50 number configurable? I like the idea of queuing all e-mails as an option. I think sendgrid sometimes introduces a bit of lag.

Yes, this is handled via the NOTIFICATIONS_PER_BATCH constant.

 

Link to comment
Share on other sites

This has been covered a few times and if you are using SMTP, it can be slower. Personally, I would task them all out once per minute. Would be nice if in ACP there was a setting/option to do this.

I'm using Amazon SES and now using the API which is faster than the SMTP method when replying to topics. 

 

17 minutes ago, Daniel F said:

Yes, this is handled via the BULK_MAILS_PER_CYCLE constant.

I think this is for bulk mail and not the amount of followers, correct? 

Link to comment
Share on other sites

19 minutes ago, Daniel F said:

Yes, this is handled via the BULK_MAILS_PER_CYCLE constant.

I thought this was the number of e-mails sent in each call to the email handler, and not for the logic that I thought @bfarber was referring to which suggests if there's less than 50 followers, the e-mails are sent immediately; otherwise the e-mails are queued.

Half paying attention in meetings with no access to source code at the moment, but weird choice if the constant applies to both those scenarios.

Link to comment
Share on other sites

4 minutes ago, CoffeeCake said:

I thought this was the number of e-mails sent in each call to the email handler, and not for the logic that I thought @bfarber was referring to which suggests if there's less than 50 followers, the e-mails are sent immediately; otherwise the e-mails are queued.

Half paying attention in meetings with no access to source code at the moment, but weird choice if the constant applies to both those scenarios.

Sorry, it was the wrong constant! I have edited my post in the meanwhile.

It's NOTIFICATIONS_PER_BATCH which handles this.

 

Sorry again for the confusion. 

Link to comment
Share on other sites

Thanks... snuck into source code while totally paying attention in this Zoom.

secret smell GIF

So, setting that constant to 0 would result in forcing all notifications/emails to be queued. The default value is 30, not 50.

I think we might just try this out. 🙂

Link to comment
Share on other sites

1 minute ago, CoffeeCake said:

So, setting that constant to 0 would result in forcing all notifications/emails to be queued. The default value is 30, not 50.

Interesting, I thought that was for bulk mail only and notification emails were different.

Link to comment
Share on other sites

I do not use Elasticsearch, and I no longer use any plugins.

I do indeed use SMTP, and the server is indeed on the slow side - each message runs through an antivirus and a spam filter before allowing send, and the server could use a bit more RAM.

I've added this to constants.php and will let yall know if things improve:

\define( 'NOTIFICATIONS_PER_BATCH', 5 );

Thanks!

By the way... Instead of basing it purely on the number of messages, wouldn't it be better to also have a NOTIFICATION_TIME_PER_BATCH, and if after sending each message, it checked if the total time had been exceeded, and if so, queue the remaining messages? That way if it's a slow server, if it takes over 3 seconds (for example), it will automatically batch without any user configuration?

Link to comment
Share on other sites

37 minutes ago, James25 said:

I've added this to constants.php and will let yall know if things improve:

\define( 'NOTIFICATIONS_PER_BATCH', 5 );

This checks to see how many notifications should be sent out, and also how many e-mails will be sent out at a single run of the background task. So if you have 4 or less people asking to be notified, it will do it immediately. If greater, it will be queued to a background task that runs with your task jobs. Setting this too low, as it is implemented now, could have undesirable consequences as instead of sending out 30 notifications per task run, it would only do five. If you set this too low, it could take quite a bit of time to process those e-mails as those notifications from all threads are going to get queued.

I would not set this option today and leave it at the default (or set it higher if your setup can handle it).

The issue is that the time it takes to send out those notifications for the first four emails may cause hanging as the communication happens with your email solution. This doesn't matter if it's a task happening in the background, but when it's immediate, it impacts the user experience by causing a delay in the time from clicking submit to the redirect to view the thread.

I'd like to see a QUEUE_ALL_NOTIFICATIONS constant or similar to turn off the immediate e-mail bit and send everything to queue.

Edited by CoffeeCake
Link to comment
Share on other sites

Going further down into this hole, things like notifications for content that needs to be approved and quotes and mentions are all immediate and exist outside of all of this.

If a poster is quoting a bunch of people and/or using @ mentions you could end up with all sorts of e-mails going out on pressing submit.

rabbit hole GIF

Edited by CoffeeCake
Link to comment
Share on other sites

I'm wondering (and I think I've wondered this before) if using a local mta on the server that then talks to SendGrid separately from IPS may be a workaround.

IPS -> Local MTA -> SendGrid

Then the MTA just collects everything instantly and handles the sending off to SendGrid in the background.

See the Mail Servers section here:

https://sendgrid.com/docs/for-developers/sending-email/

Link to comment
Share on other sites

 Share

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...