sendgrid integration

[LiveG]

Hi,

I want to integrate SendGrid as a mail relay service for my forum.

Can this service be used for inbound&outbound mail service?

Is there any deployment guide? i have opened a free SendGrid account for testing and activated the service and API key on the ACP. But it doesn't work correctly.

Many thanks.

Edited April 7, 2023 by LiveG

[Randy Calvert]

Receiving email is not applicable to IPB. It’s only for sending. 

There is more to sending email than just getting an API key. You need to setup SPF and DKIM, etc. Those are all things you would do directly inside of Sendgrid not related to IPB. You would want to work with them on that side first before trying to send mail otherwise most mail providers will reject the messages from them. 

[LiveG]

Thank you.

So what is the best solution for outbound&unboung mail?

Google keeps blocking me.

 

[Marc]

The first thing you need to do these is to ascertain why google is blocking you. WIthout an answer to that question, its difficult to really answer you

[LiveG]

10 minutes ago, Marc Stridgen said:

The first thing you need to do these is to ascertain why google is blocking you. WIthout an answer to that question, its difficult to really answer you

I have reached Google's service limitation.

https://support.google.com/a/answer/166852?hl=en

Edited April 11, 2023 by LiveG

[Marc]

Ah, indeed that would be an issue. The only suggestion I can make is to use a different provider that suites your email numbers. We have sendgrid integration for example, which may be better suited

[LiveG]

11 minutes ago, Marc Stridgen said:

Ah, indeed that would be an issue. The only suggestion I can make is to use a different provider that suites your email numbers. We have sendgrid integration for example, which may be better suited

SendGrid will solve for me only the outgoing mails; what about the incoming? 

Does the ACP support configuration for two mailboxes?

[Randy Calvert]

You can still use Google for sending and receiving email as normal.  Sendgrid can be setup by using SPF and DKIM to ALSO be able to send. 
 

You don’t need two mailboxes. 

IPB would send through Sendgrid. (Your outbound listed address may be admin@yourdomain.com). If someone replies, the reply would go to Google which would be the receiving server MX.  IPB does not need to be aware of INBOUND email. 

Edited April 11, 2023 by Randy Calvert

[Editorn]

On 4/11/2023 at 5:42 PM, Randy Calvert said:

IPB does not need to be aware of INBOUND email. 

Unless used with Support Requests in Commerce 😉

[LiveG]

On 4/11/2023 at 6:42 PM, Randy Calvert said:

You can still use Google for sending and receiving email as normal.  Sendgrid can be setup by using SPF and DKIM to ALSO be able to send. 
 

You don’t need two mailboxes. 

IPB would send through Sendgrid. (Your outbound listed address may be admin@yourdomain.com). If someone replies, the reply would go to Google which would be the receiving server MX.  IPB does not need to be aware of INBOUND email. 

Google keeps blocking me.

If someone can guide and help me with this SendGrid integration(outgoing mail) and Google (incoming) , i am willing to pay

please quote me in private

Edited June 30, 2023 by LiveG

[Randy Calvert]

Blocking what?  Mail from sendgrid, mail from your personal email on the domain?  

If it’s the mail sent through Sendgrid look at the logs in Sendgrid to see why. If you are using a Sendgrid shared IP, it could just be a reputation issue. You may need a dedicated IP.  I quit using Sendgrid after roughly 20 percent of mail could not be delivered to Hotmail and Comcast users because Sendgrid’s IP addresses were being rejected by them. 

[LiveG]

Mail from my personal forum domain is blocked. (hundreds of emails are going out from my forum per day, this is the reason)

I don't know how to integrate IPB with my SendGrid account for outgoing mail and make the incoming mail go via Google.

So complicated.

Edited June 30, 2023 by LiveG

[Randy Calvert]

Incoming mail is controlled by MX records in DNS.  

For example, IPS also uses Google for receiving email:

admin@localhost ~ % dig invisioncommunity.com MX

; <<>> DiG 9.10.6 <<>> invisioncommunity.com MX
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3130
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 2

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;invisioncommunity.com.		IN	MX

;; ANSWER SECTION:
invisioncommunity.com.	300	IN	MX	5 alt1.aspmx.l.google.com.
invisioncommunity.com.	300	IN	MX	5 alt2.aspmx.l.google.com.
invisioncommunity.com.	300	IN	MX	1 aspmx.l.google.com.
invisioncommunity.com.	300	IN	MX	10 aspmx2.googlemail.com.
invisioncommunity.com.	300	IN	MX	10 aspmx3.googlemail.com.

That routes inbound email to Google's various servers.  

Now...  for OUTBOUND email, you have to list both Google and Sendgrid in SPF/DKIMM.  

In looking at the Invision SPF record, you see:

v=spf1 mx include:servers.mcsv.net include:_spf.google.com include:sendgrid.net include:mail.zendesk.com include:helpscoutemail.com -all

Each service provider needs to be included with an "include:" statement.  

For DKIM records, you need to publish both Google and SendGrid's record via DNS as they instruct.  And yes, you can have multiple DKIM records.

https://powerdmarc.com/multiple-dkim-records-on-email-domain/#:~:text=To publish multiple DKIM records,concatenated into your previous records.

When you say email from your server is blocked, I assume it's Sendgrid that is sending that email.  Look up the email in the Sendgrid console and see what it says for the message.  Why is the other mail server rejecting it?  If it's a configuration with SPF/DKIM, it should say that.  It might also simply be that you're using Sendgrid's shared IPs and they are being blacklisted.  (In that case, you would need a dedicated sending IP from them.)

[LiveG]

Thank you @Randy Calvert

For the Inbound traffic,

I have checked the MX record for my domain on this site https://mxtoolbox.com/

And saw several MX Google servers answering my domain :

Your email service provider is "Google Apps"

But with some errors https://pasteboard.co/0cSS5AdrxEA5.png

What should i do? and now for the actual mailbox under this domain, where should I create it? on my forum server?

Edited July 2, 2023 by LiveG

[LiveG]

i will clarify,

once someone's contact form email reaches the Google mail server specified in my MX records, the mail server will typically forward the email to my forum server? if not, how do i make google to forward all the emails to my forum? any setup and configuration needed?

 

Edited July 2, 2023 by LiveG

[Randy Calvert]

2 hours ago, LiveG said:

But with some errors https://pasteboard.co/0cSS5AdrxEA5.png

DMARC is not a real issue.  In fact, Invision does not have DMARC enabled either and their mail delivers fine.

1 hour ago, LiveG said:

once someone's contact form email reaches the Google mail server specified in my MX records, the mail server will typically forward the email to my forum server? if not, how do i make google to forward all the emails to my forum? any setup and configuration needed?

 

If someone fills out a Contact form on your site (and you're configured to send that message as email instead of routing to a support request in Commerce app), that is an OUTBOUND from your site via Sendgrid to wherever you have your have your emails pointed to.  If it's pointed to an email address that is on your domain (like webmaster@domain.com),  Sendgrid does a MX lookup on the domain and receives Google's MX and sends there.  It does not come back to your forum.

IPB does not have a concept of importing emails into your forum.  (The Commerce app can fetch support tickets via POP3, but that's the only sort of way of getting email into IPB.). You can't send an email for example and have it post a topic or reply to a topic.  

Think of it this way...  let's say you have the domain "yourdomain.com".  On IPB you send email as coming from "webmaster@yourdomain.com".  

With Sendgrid setup, any email sent comes from webmaster@yourdomain.com.  But you also have webmaster@yourdomain.com setup in Google....  you can send AND receive email from webmaster@yourdomain.com.  That means if someone replies to the email sent from IPB, the reply will end up in your Gmail because it's the INBOUND MX record for your domain.  

Sendgrid = Outbound email only
Google = Inbound AND outbound

Sendgrid sends mail on your behalf.  It cannot receive anything.  There is no way to have email come INBOUND into your forum.  It's "exit only".  

[LiveG]

Thank you, Randy!!! It's very clear to me now.

Does ipb have any sandgrid setaup menual guide?

Edited July 2, 2023 by LiveG

[LiveG]

And regarding the webmaster@yourdomain.com mailbox in your example, is this mailbox need to be created on my server forum site? With which Exchange server?

I can't create it via Google because they are blocking this mailbox, as it sends a lot of emails

Edited July 2, 2023 by LiveG

[Randy Calvert]

8 hours ago, LiveG said:

And regarding the webmaster@yourdomain.com mailbox in your example, is this mailbox need to be created on my server forum site? With which Exchange server?

I can't create it via Google because they are blocking this mailbox, as it sends a lot of emails

You don’t have to create the email address.  Sendgrid can send as ANYTHING you want.  If you don’t create the address however, users won’t be able to reply to the email and have it go anywhere that you see the response. 

On this site, IPB uses the outgoing address noreply@invisionpower.com to help encourage/remind people that if they reply to an email notification that it won’t be seen.  

There is no right or wrong answer… it just depends on how you want your community to operate.  On my site, emails are sent from moderators@yourdomain.com… or you could do something like community@ or notify@.