Slimer Posted January 24, 2017 Posted January 24, 2017 Many users has bad email: account deleted, user suspended, no space in mailbox etc. Your mails to this users back to you with errors. With this hook user MUST be to change his email and confirm it. Letters to selected users did not sent and you have no errors. How to get users list with bad emails? 1. Manually: see your mailer-daemon letters 2. Auto: in mail error.log i am doing like this: #BAD EMAILS cat mail.log|grep "user is terminated\|No such user\|account is disabled\|The email account that you tried to reach does not exist" >mail.log.badmails cat mail.log.badmails|cut -d " " -f1,2|uniq -c >mail.log.badmails.stats sed -E "s/[[:space:]]+/ /g" mail.log.badmails|cut -d " " -f7|sort|uniq -c|sort -n -r|sed 's/to=<//g'|sed 's/>,//g'|sed 's/ //g' >mail.log.badmails.to.stats sed -E "s/[[:space:]]+/ /g" mail.log.badmails|cut -d " " -f7|sort|uniq -c|sort -n -r|sed 's/.*to=<//g'|sed 's/>,//g' >mail.log.badmails.list You get file mail.log.badmails.list where all your bad mails listed. Just copy-paste into config. Thats all.
MediaDiGi Posted January 24, 2017 Posted January 24, 2017 Hello You have a nice piece of work here. In fact, I was working at something like this but I dont have knowledge of IPS framework. That's why I developed the other part: using free PHP libraries the php script is connecting to email server using POP3 or IMAP, scan received emails AND return a list of bad emails. I can give reference to those libraries.. to add them to your plugin so will work automatically like this: After admin send an email/newsletter the plugin with a hook scan the email server for returned(bad)emails and automatically change those users status to validating or something.
Janyour Posted January 24, 2017 Posted January 24, 2017 Looks awesome, how its working your app ? can you explain more please ?
Slimer Posted January 26, 2017 Author Posted January 26, 2017 On 24.01.2017 at 9:47 PM, MediaDIGI.com said: php script is connecting to email server using POP3 or IMAP, scan received emails AND return a list of bad emails Where you have many users, additional connections unwanted. A have problems with SPAM services, that blocked my server IP becouse there are a lot of letters returned, no existing users (like brooteforce mailing), other errors. Thats why I desided to block all users with bad emails and reduce amount of mails sended. Also spamers uses "one-day" emails for approving their accounts. On 24.01.2017 at 10:42 PM, Janyour said: can you explain more please ? Then user logged in - its checking for existing in bad-emails list. If his mail is on the list - user blocked and must change email.
Cyboman Posted January 26, 2017 Posted January 26, 2017 Good work, but requires some enhancements! Bad emails are a pain in the a**! Nowadays your community gets shutout from big email providers if you send newsletters very rapidly megafast! F.e., I have a community with more than 200.000 member. I could imagine 1/3 of the total registrations has bad email adresses. I need a solution! I want to restart my newsletters! Facts are: every 3rd or 4th account has become a bad email during past 15 years, using temporary mailboxes, obsolete adresses, fake adresses a.s.o. you are not allowed to send emails to members in europe, that haven't opted in willingly (this only happens during the registration process, if a new user clicks "send me news and information" or "allow emails by admin" (former phrase in IPB3)). Most plugins don't respect this enormous important option, that must be guaranteed. If I try "mailbouncer" with sparkpost, I reckon I will be banned straight ahead while sending the very first newsletter. If I use "Force users to change email" I will be banned too by Sparkpost or Mailchimp and my reputation will be gone forever. If I use any of above tools with SMTP mailout via my local server, "mailbouncer" won't work. And your plugin won't help me, if I get blacklisted afterwards by Gmail, Hotmail ... due to the fact that member don't remember they once registered and decided to receive emails by admin. What can I do? I think this is the right solution: Your plugin points into the right direction, as well as mail bouncer plugin, but it's missing some essential features: A plugin, that's made to get rid off bad emails, should randomly send only a few "test emails" to only a small member circle every day, maybe with "do you still need your account? please come back" and see if the email is still good. F.e. every day only 50-100 members, so there won't be a shockwave for providers. If the email is bad, the plugin should (1) deactivate the email for bulk emails AND (2) deactivate the checkmark for receiving emails at all in the members notifications settings. This is the only way to get a clear database in a smooth manner. But not only randomly sent emails should be taken for decision. Also regular bulk emails have to be checked continuously if there are bad emails, and f.e. if there are 500 bad emails detected or f.e. a bad quota (more than 3 %) than the plugin should pause the sending process. And restart another day... By the way, I think it still is too complicated to "generate manual lists" that have to be inserted somewhere... The whole process has to become automatic. And members should be canceled from receiving any more emails at all, if the email isn't working and they opted in for receiving mails before! Anyone has a suggestion for me please?
Dave Baker Posted January 26, 2017 Posted January 26, 2017 Slimer, I have downloaded your plug-in -- it is working very well! Can you please tell me how I can change the Russian text into English text, on the first modal box (the one with "Change Email Address" and "Sign Out" buttons).
MediaDiGi Posted January 26, 2017 Posted January 26, 2017 @Cyboman I feel you buddy. Been in your shoes. banned by Sparkpost, then sent an email with my own server and got the ip blacklisted as 1/4 or 1/3 of my member emails become inexistent. Im looking for a plugin that will solve the issue. Ive already sent a mass newsletter so now I got a buch of returned emails that I filtered and entered in the program but the downside is that I got blacklisted.. However I can get delisted from the spam lists easily. If you dont want to send newsletters and first to verify emails you can use http://email-verify.my-addr.com/online-email-verification-validator-api.php They charge like 20 cents per 1000 emails to be verified. However I also vote for a one stop shop solution that will do everything you pointed out.
Slimer Posted January 27, 2017 Author Posted January 27, 2017 @Cyboman, @MediaDIGI.com i have same problems and understand you. With very big "bad emails list" this can slow your site becouse every mail sending it's compare with this list. Now i'm working on new solution. May be it's be better - will see. On 26.01.2017 at 9:43 PM, Cyboman said: Bad emails are a pain in the a**! Did not understand this. On 26.01.2017 at 9:43 PM, Cyboman said: If I use "Force users to change email" I will be banned too by Sparkpost or Mailchimp and my reputation will be gone forever. Why? If you use my hook - you will send mails only to existing emails (new email, that member entered). On 26.01.2017 at 9:43 PM, Cyboman said: By the way, I think it still is too complicated to "generate manual lists" that have to be inserted somewhere... Maybe, but if you have 200K users you must have server administrator, which can help. I have given a code which should be executed on your server to get all your bad emails. Two seconds of work and you not need to send any letters, use other checking services etc. You can do this procedure 2-4 times a year and 99% of bad mails won't be send. 20 hours ago, MediaDIGI.com said: However I also vote for a one stop shop solution that will do everything you pointed out. Miracles don't happen))
Slimer Posted January 27, 2017 Author Posted January 27, 2017 22 hours ago, Dave Baker said: Can you please tell me how I can change the Russian text into English text For editing this page go to your theme editor and edit template: core - global - plugins - notValidated2 Change all as you wish, except this buttons: <p class='ipsType_normal'> <a href='{url="app=core&module=system&controller=register&do=changeEmail" seoTemplate="register"}' data-ipsDialog data-ipsDialog-size='narrow' data-ipsDialog-title='{lang="reg_change_email"}' data-ipsDialog-modal='true' class='ipsButton ipsButton_light ipsButton_verySmall'>{lang="reg_change_email"}</a> <a href='{url="app=core&module=system&controller=login&do=logout" csrf="true" seoTemplate="logout"}' class='ipsButton ipsButton_light ipsButton_verySmall'>{lang="sign_out"}</a> </p> File reuploaded. You can download new version with english text.
Dave Baker Posted January 30, 2017 Posted January 30, 2017 Slimer, when the member has confirmed the change in email address by clicking on the button in the confirmation email, the plug-in sends an email to the old email address -- see the attached image -- but the email is being sent to "0" instead of sending it to the old email address. Could you please change the plug-in to send the email to the old email address? Also, I was going to suggest that you eliminate that email completely -- because we know it will bounce back, since it's going to the old (bad) email address -- but it's a nice thing for the Admin to receive the email when it bounces back, because the email shows the old and new email addresses, so the Admin has a record (a kind of a log) of what's happened.
Slimer Posted January 30, 2017 Author Posted January 30, 2017 1 hour ago, Dave Baker said: I was going to suggest that you eliminate that email completely Email to old address now do not sending. Download version 1.0.1. Thanks for the report! We must eliminate maximum bounced emails not to get in the black list.
Dave Baker Posted January 30, 2017 Posted January 30, 2017 Here are the contents of my template for the first page that the member sees, advising him or her of the need to change the email address, in case it's useful to other people ( Admin CP -> Customization -> Themes -> </> -> templates -> core -> global -> plugins -> notValidated2 ): <section class='ipsType_center ipsPad'> <br><br> <i class='ipsType_huge fa fa-envelope'></i> <h1 class='ipsType_veryLarge'>Please Change Your Email Address</h1> <p class='ipsType_large'> Email to this address is being returned as undeliverable:<br><b>{{$mail = \IPS\Member::loggedIn()->email;}}{$mail}</b> </p> <p class='ipsType_large'> In order for you to receive email notifications about topics or forums you're following, a working email address is needed. Please change your email address. </p> <hr class='ipsHr'> <p class='ipsType_normal'> <a href='{url="app=core&module=system&controller=register&do=changeEmail" seoTemplate="register"}' data-ipsDialog data-ipsDialog-size='narrow' data-ipsDialog-title='{lang="reg_change_email"}' data-ipsDialog-modal='true' class='ipsButton ipsButton_light ipsButton_verySmall'>{lang="reg_change_email"}</a> <a href='{url="app=core&module=system&controller=login&do=logout" csrf="true" seoTemplate="logout"}' class='ipsButton ipsButton_light ipsButton_verySmall'>{lang="sign_out"}</a> </p> </section>
Cyboman Posted January 30, 2017 Posted January 30, 2017 On 27.01.2017 at 0:05 AM, MediaDIGI.com said: If you dont want to send newsletters and first to verify emails you can use http://email-verify.my-addr.com/online-email-verification-validator-api.php They charge like 20 cents per 1000 emails to be verified. If such services could be included in any way, so it would have a direct connection on the IPS database, that would be really great. Maybe also in a different plugin ("idea for author" :)) !
MediaDiGi Posted January 31, 2017 Posted January 31, 2017 4 hours ago, Cyboman said: If such services could be included in any way, so it would have a direct connection on the IPS database, that would be really great. Maybe also in a different plugin ("idea for author" :)) ! Totally agree. I also suggest the author think about the possibility to create a solution that solves all these matters. A quite similar application but for those that use sparkpost and mandrill priced at $35 per copy + $15 renewal was sold 99 times.. You should do the math So @Slimer there are money to be made if you have the time to develop a complete solution for email bouncing.
Dave Baker Posted January 31, 2017 Posted January 31, 2017 Slimer, I love this plug-in! I also need a way to know when the member has changed his or her email address. (I then move the member from a "Email Needs to be Changed" group back to the Registered group. By having an "Email Needs to be Changed" group, I can be sure that I don't send a bulk email to bad email addresses, because any bulk email is sent only to the Registered group and not to the Email Needs to be Changed group.) Could you add an email to the Admin when the member has changed his or her email address, or some kind of log that the Admin could check from time to time? Either one would be so helpful. I believe another author's plug-in will nag a member via a modal pop-up until the member has changed his or her bad email address, so maybe there is some code in that plug-in that could be useful to you. I have not installed that plug-in, so I do not know if it correctly works to stop the pop-ups when the member has changed his or her bad email address. Ideally the email to the Admin or the log would show the name of the member. But just having the old email address and the new address would be good enough because the Admin could search the members and find the name of the member. I would be happy to help pay for the development of this feature. Thank you!
Dave Baker Posted February 1, 2017 Posted February 1, 2017 We find out about bad emails from a log that is kept by SMTP.com, which is the service we use to send out our emails. (Postfix on our server connects directly with SMTP.com.) The log shows bounces that come back to SMTP.com. We are very content with assembling a list of the bad emails manually, and would not need to have a plug-in automatically handle them. We are happy to manually add the most recently bouncing emails into the list that we previously saved into the form that pops up when clicking on the Edit link for your plug-in. Thanks!
Cyboman Posted February 1, 2017 Posted February 1, 2017 17 minutes ago, Dave Baker said: We are very content with assembling a list of the bad emails manually, and would not need to have a plug-in automatically handle them. We are happy to manually add the most recently bouncing emails into the list that we previously saved into the form that pops up when clicking on the Edit link for your plug-in. @Dave Baker, sorry, I think you didn't get the above discussion point. Your comment, what feature "you don't require", isn't really constructive for others. I believe it's easy for every small community admin, who has 2500 members or alike in his community, this user amount is easily manageable. You could also edit member accounts manually without this tool, and just toggle the "send me news and informations" option in member profiles, after you identified the bad emails (and you won't get blacklisted, because your member count is that small!) But... (1) we were talking about a mechanism, before you start bulk emailing at all, to identify bad emails before the first bulk mail sendout. Because you will be blacklisted directly with tens of thousands bad emails. My situation / case: There is no historic email error log for formerly active members ages ago, as we changed servers serveral times. Members being inactive f.e. for 2 years, won't be included. I want to use the plugin for mailing ALL member, who opted in for "send me news and informations", for a very first time. Hence it requires a precheck before first sendout. (2) I doubt that if you try to insert more than 50.000 or more lines with email addresses in the editor ("list of recently bouncing emails"), that this will work. I think the editor will freeze. But I admit, I haven't tested this so far... Don't understand me wrong. I highly appreciate every email solution for IPS, as this is a very big problem for a lot - especially big - communities. @Slimer, thumbs up, you've definitely done great work with your plugin! I just try to convince you, that above problems still exist. And I would also be willing to pay a lot for a professional email solution, that guarantees me a valid email address base in a way, before I start the first bulk mail sendout, to avoid being blacklisted directly. Thank you!
Dave Baker Posted February 1, 2017 Posted February 1, 2017 Well, no, Cyboman, my response to SLimer is "not constructive for others" to the extent you're wanting SLimer to do something the plug-in wasn't written to do, such as the toasting of bagels. As MediaDigi.com said earlier, use a commercial service and drop into their form a list of all of your registered users' email addresses, which you can export from the Members search feature in the AdminCP. This is a terrifically simple and effective plug-in when a list of bad email addresses is known.
Slimer Posted February 1, 2017 Author Posted February 1, 2017 On 31.01.2017 at 7:57 PM, Dave Baker said: Ideally the email to the Admin or the log would show the name of the member. But just having the old email address and the new address would be good enough because the Admin could search the members and find the name of the member. I would be happy to help pay for the development of this feature. ok, i think about this. 1 hour ago, Cyboman said: Hence it requires a precheck before first sendout. you need to use special services/programms, then you need, for example, to move all bad users to the new group (i think about this feature too). And then send mails to all, except this group.
Slimer Posted February 6, 2017 Author Posted February 6, 2017 Hello everyone! Good news: I am completly rewrite this plugin and get new one: Now with log, ability to move users to another group and new type of work: now you do all actions once and all users with bad emails same time became unapproved. Even if they are already logged in. @MediaDIGI.com, @Dave Baker write me pm, this version free of charge for you.
Dave Baker Posted February 13, 2017 Posted February 13, 2017 Wonderful! I am happy to pay for the new plugin, and am going to buy, download and install it now. Thanks so much.
Recommended Posts
Archived
This topic is now archived and is closed to further replies.