So here's my code… It comes with a lot of caveats… Think of it as starting point so you or your developer doesn't have to start from scratch. I'm not the best PHP programmer, so my code may not be quite as beautiful as yours, but it's better than nothing. Also, this code calls MailBouncer, so you must have an active license for that. You can put the the file anywhere and name it anything (ending in .php). The code will almost certainly need to be tweaked. For example, my mail server is running SendMail. If your mail server is running something different the bulk of your messages may have different formatting.
Here's what it does… Logs into your email account and scans through all the messages from postmaster@ and MAILER-DAEMON@. You can login with IMAP or POP w/ or w/o SSL. Only IMAP w/o SSL has been tested, but the code is there for the others. After processing the entire mailbox, when you repeat the process you'll probably want to do just examine the most recent period. To do that add ?days=X to the end of the URL where X is the number of days you want to examine. X can be a decimal number.
When it scans it classifies things as 4 types:
Indeterminate
Reputation problem
Hard bounce
Soft bounce
You won't see detailed information about hard and soft bounces, but you will if there's a reputation problem or the outcome was indeterminate.
Reputation problems typically mean that your server got on a blacklist. You'll want to rectify this ASAP. Reputation problems are handled as soft bounces, since they're not the fault of the member.
Indeterminate issues are when you'll probably need to tweak the code to handle the case (typically different error formatting). You can add trigger phrases to $softphrases, $hardphrases and $reputationphrases. If it's not parsing the email address that's bouncing, that's a little trickier. Or you can just manually go into the IP board admin area and do manually what Mail Bouncer does.
It is possible to change up the code to not scan a mailbox and just handle a long list of emails you've manually put together – just replace everything inside
if (!$mbox = imap_open($authhost, $user, $pw)){ } else {}
with something that populates the $hardfails and $softfails arrays.
At the end it does one other thing – it queries the database and finds everyone who has been banned for 8 weeks or longer and processes their email addresses as hard bounces. These people are probably mad at you and don't want your emails, and they're likely to mark your emails as spam just to spite you. So it's best to just treat the email as bad and if/when the member returns they can revalidate their email address. The SQL query is shown. You'll need to rewrite the actual pull from your IP.Board database. I use Fat Free Framework, so that's the syntax you'll see. To turn off handling of long bans permanently just comment out or delete the code. But you can also add lb=off to the end of the URL to not to it on a particular run of the script.
With all of that said, here's the code…
<?php
//I USE FAT FREE FRAMEWORK, SO THERE'S AT LEAST ONE CASE WHERE YOU NEED TO REPLACE F3 CODE WITH YOUR OWN CODE
$user = 'your_mail_username';
$pw = 'your_mail_pw';
$mailserver = 'localhost';
\define('REPORT_EXCEPTIONS', TRUE);
$_SERVER['SCRIPT_FILENAME'] = __FILE__;
require_once '/path/to/init.php'; //PATH TO THE INIT.PHP FILE FOR YOUR IP.BOARD INSTALL
header('Content-Encoding: none');
ob_implicit_flush(true);
ob_end_flush();
set_time_limit(60*180);
echo str_repeat(' ',1024*64.1);
$hardfails = array();
$softfails = array();
//THE FOLLOWING ARRAYS OF PHRASES DETERMINE THE TYPE OF FAILURE, EDIT THEM AS NEEDED
$softphrases = array();
$softphrases[] = '(reason: 552'; //552 = Exceeded storage allocation
$hardphrases[] = '<<< 552';
$softphrases[] = ' over quota';
$softphrases[] = ' inbox is full';
$softphrases[] = ' mailbox is full';
$softphrases[] = ' mailbox full';
$softphrases[] = 'Mail quota exceeded';
$softphrases[] = 'is over quota';
$hardphrases = array();
$hardphrases[] = '550 5.1.1';
$hardphrases[] = '550-5.1.1';
$hardphrases[] = '<<< 550'; //550 = Non-existent email address
$softphrases[] = '(reason: 550';
$hardphrases[] = 'Status code: 550';
$hardphrases[] = ' mailbox unavailable';
$hardphrases[] = ' deactivated mailbox';
$hardphrases[] = ' Bad destination mailbox address';
$hardphrases[] = ' no valid recipients';
$hardphrases[] = 'Not a valid recipient';
$hardphrases[] = 'User unknown';
$hardphrases[] = ' Address does not exist';
$hardphrases[] = ' mailbox is disabled';
$hardphrases[] = ' user doesn\'t have a yahoo';
$hardphrases[] = 'is a deactivated mailbox';
$reputationphrases = array(); //things that indicate they just don't like/trust the email
//this is a problem if you sent the email, not a problem if some spammer sent the email (pretending to be you)
$reputationphrases[] = 'Relay access denied'; //https://serversitters.com/how-to-correct-554-5-7-1-relay-access-denied-email-errors-and-prevent-them-in-the-future.html
$reputationphrases[] = ' blocked using ';
//ONE OF THE FOLLOWING 4 LINES SHOULD BE UNCOMMENTED DEPENDING ON WHAT TYPE OF CONNECTION YOU WANT TO MAKE TO YOUR MAIL SERVER
//$authhost="{".$mailserver.":995/pop3/ssl/novalidate-cert}"; //POP w/o SSL
//$authhost="{".$mailserver.":110/pop3/notls}"; //POP w/ SSL
$authhost="{".$mailserver.":993/imap/ssl/novalidate-cert}"; //IMAP w/o SSL (this is the only one that's been tested)
//$authhost="{".$mailserver.":143/imap/notls}"; //IMAP w/ SSL
echo '<!DOCTYPE html><html lang="en-US" dir="ltr"><head><meta charset="utf-8"></head><body>';
if (!$mbox = imap_open($authhost, $user, $pw)){
echo "<p>Could not connect to mail server.</p>";
} else {
echo "<p>Connected to mail server.</p>";
$totalrows = imap_num_msg($mbox);
echo "<p>Found $totalrows messages</p><ul>";
$result = imap_fetch_overview($mbox, "1:$totalrows", 0);
foreach ($result as $overview) {
$body = imap_fetchbody($mbox, $overview->msgno, "1");
$datetime = strtotime($overview->date);
date_default_timezone_set("UTC");
if (stripos($body, 'https://web.de/email/senderguidelines') !== false && stripos($body, 'cs2172.mojohost.com') !== false) {
//ignore these, problem was resolved
} else if ($_GET['days'] > 0 && $datetime < time() - ($_GET['days'] * 86400)){
//do nothing, it doesn't meet the cutoff
// echo "<li>Skipping: $datetime < ".(time() - ($f3->get('days') * 86400))."</li>";
} else if ((stripos($overview->from, 'Mailer-Daemon@') !== false || stripos($overview->from, 'postmaster@') !== false )
&&
((strpos($body, '----- The following addresses had permanent fatal errors -----') !== false && strpos($body, '(reason: ') !== false) ||
(stripos($body, 'Final-Recipient: ') !== false) ||
(stripos($body, 'message couldn\'t be delivered to ') !== false) ||
(stripos($body, '> is a deactivated mailbox') !== false) ||
(strpos($body, ' to these recipients or groups:') !== false && strpos($body, 'The recipient\'s mailbox is full') !== false))) {
//PARSE THE EMAIL ADDRESS THAT'S HAVING A PROBLEM
$bademail = '';
if (strpos($body, '----- The following addresses had permanent fatal errors -----') !== false) {
$pos1 = strpos($body, '----- The following addresses had permanent fatal errors -----');
$pos1 = $pos1 + strlen('----- The following addresses had permanent fatal errors -----');
$pos2 = strpos($body, '(reason: ', $pos1);
if ($pos2 > 0) {
$bademail = str_replace('<', '', str_replace('>', '', trim(substr($body, $pos1, $pos2 - $pos1))));
}
} else if (stripos($body, 'Final-Recipient: ') !== false){
//The syntax of the field is as follows:
// "Final-Recipient" ":" address-type ";" generic-address
// i.e. Final-Recipient: rfc822; skijanje-zg@net.hr
$pos1 = strpos($body, 'Final-Recipient: ');
$pos1 = $pos1 + strlen('Final-Recipient: ');
$pos2 = strpos($body, "\r", $pos1);
if (strpos($body, "\n", $pos1) < $pos2){
$pos2 = strpos($body, "\n", $pos1);
}
if ($pos2 > 0) {
$bademail = substr($body, $pos1, $pos2 - $pos1);
$pos3 = strpos($bademail, ';');
if ($pos3 !== false){
$bademail = substr($bademail, $pos3 + 1);
}
$bademail = trim($bademail);
}
} else if (strpos($body, ' to these recipients or groups:') !== false && strpos($body, 'The recipient\'s mailbox is full') !== false){
$pos1 = strpos($body, ' to these recipients or groups:');
$pos1 = $pos1 + strlen(' to these recipients or groups:');
$pos2 = strpos($body, 'The recipient\'s mailbox is full', $pos1);
if ($pos2 > 0) {
$bademail = trim(substr($body, $pos1, $pos2 - $pos1));
if (strpos($bademail, '<') !== false && strpos($bademail, '>') !== false){
$bademail = substr($bademail, strpos($bademail, '<') + 1);
$bademail = substr($bademail, 0, strpos($bademail, '>'));
}
}
} else if(stripos($body, '> is a deactivated mailbox') !== false){
$bademail = substr($body, 0, stripos($body, '> is a deactivated mailbox'));
$bademail = substr($bademail, strrpos($bademail, '<') + 1);
} else if(stripos($body, 'message couldn\'t be delivered to ') !== false){
$bademail = substr($body, stripos($body, 'message couldn\'t be delivered to ') + strlen('message couldn\'t be delivered to '));
$bademail = substr($bademail, 0, strpos($bademail, '. '));
}
//CLEAN UP THE EMAIL ADDRESS
if (stripos($bademail, 'mailto:') === 0){
$bademail = substr($bademail, strlen('mailto:'));
}
if (!filter_var($bademail, FILTER_VALIDATE_EMAIL)) {
echo "<li><b>ERROR: PARSED THE EMAIL $bademail BUT IT IS NOT A VALID EMAIL ADDRESS</b></li>";
$bademail = '';
}
//EVALUATE DIFFERENT TYPES OF PROBLEMS
$result = 'Indeterminate';
if ($result == 'Indeterminate'){ //reputation problems
foreach($reputationphrases as $phrase){
if ($result == 'Indeterminate' && stripos($body, $phrase) !== false) {
$result = 'Reputation Problem';
}
}
}
if ($result == 'Indeterminate'){ //hard bounces
foreach($hardphrases as $phrase){
if ($result == 'Indeterminate' && stripos($body, $phrase) !== false) {
$result = 'hard';
}
}
}
if ($result == 'Indeterminate'){ //soft bounces
foreach($softphrases as $phrase){
if ($result == 'Indeterminate' && stripos($body, $phrase) !== false) {
$result = 'soft';
}
}
}
if ($result == 'Indeterminate' || $bademail == '') {
echo "<li>{$overview->msgno}<ul>";
echo "<li>Sent: {$overview->date}</li><li>From: " . htmlspecialchars($overview->from) . "</li><li> Subject: {$overview->subject}</li>";
echo "<li><pre>" . htmlspecialchars(substr($body, 0, 10240)) . "</pre></li>";
echo "<li><b>Bad Email: $bademail</b></li>";
echo "<li>Date/Time: $datetime</li>";
echo "<li><b>Result: $result</b></li>";
echo "</ul></li>";
} elseif ($result == 'Reputation Problem') {
echo "<li>{$overview->msgno}<ul>";
echo "<li>Sent: {$overview->date}</li><li>From: " . htmlspecialchars($overview->from) . "</li><li> Subject: {$overview->subject}</li>";
echo "<li><pre>" . htmlspecialchars(substr($body, 0, 10240)) . "</pre></li>";
echo "<li><b>Bad Email: $bademail</b></li>";
echo "<li>Date/Time: $datetime</li>";
echo "<li><b>Result: Reputation Problem (handled as a soft bounce)</b></li>";
echo "</ul></li>";
$exists = false;
for($i = 0; $i < count($softfails); $i++){
list($temp_email, $temp_dt) = explode(',', $softfails[$i]);
if ($temp_email == $bademail){
$exists = true;
if ($temp_dt < $datetime){
$softfails[$i] = "$bademail,$datetime"; //update date time
}
}
}
if ($exists === false){
$softfails[] = "$bademail,$datetime";
}
} elseif ($result == 'hard' && array_search("$bademail,$datetime", $hardfails) === false) {
$exists = false;
for($i = 0; $i < count($hardfails); $i++){
list($temp_email, $temp_dt) = explode(',', $hardfails[$i]);
if ($temp_email == $bademail){
$exists = true;
if ($temp_dt < $datetime){
$hardfails[$i] = "$bademail,$datetime"; //update date time
}
}
}
if ($exists === false){
$hardfails[] = "$bademail,$datetime";
}
} elseif ($result == 'soft' && array_search("$bademail,$datetime", $softfails) === false) {
$exists = false;
for($i = 0; $i < count($softfails); $i++){
list($temp_email, $temp_dt) = explode(',', $softfails[$i]);
if ($temp_email == $bademail){
$exists = true;
if ($temp_dt < $datetime){
$softfails[$i] = "$bademail,$datetime"; //update date time
}
}
}
if ($exists === false){
$softfails[] = "$bademail,$datetime";
}
}
} else if (stripos($overview->subject, "Mail delivery failed") !== false) {
echo "<li>{$overview->msgno}<ul>";
echo "<li>Sent: {$overview->date}</li><li>From: " . htmlspecialchars($overview->from) . "</li><li> Subject: {$overview->subject}</li>";
echo "<li><pre>" . htmlspecialchars(substr($body, 0, 10240)) . "</pre></li>";
echo "</ul></li>";
} else if (stripos($overview->subject, "Delivery Status Notification (Failure)") !== false) { //hotmale sends ones like this…
echo "<li>{$overview->msgno}<ul>";
echo "<li>Sent: {$overview->date}</li><li>From: " . htmlspecialchars($overview->from) . "</li><li> Subject: {$overview->subject}</li>";
echo "<li><pre>" . htmlspecialchars(substr($body, 0, 10240)) . "</pre></li>";
echo "</ul></li>";
} else if (stripos($overview->subject, "Undeliverable: ") !== false) { //outlook sends ones like this…
echo "<li>{$overview->msgno}<ul>";
echo "<li>Sent: {$overview->date}</li><li>From: " . htmlspecialchars($overview->from) . "</li><li> Subject: {$overview->subject}</li>";
echo "<li><pre>" . htmlspecialchars(substr($body, 0, 10240)) . "</pre></li>";
echo "</ul></li>";
} else if (stripos($overview->from, 'postmaster@') === 0) { //catch all
echo "<li>{$overview->msgno}<ul>";
echo "<li>Sent: {$overview->date}</li><li>From: " . htmlspecialchars($overview->from) . "</li><li> Subject: {$overview->subject}</li>";
echo "<li><pre>" . htmlspecialchars(substr($body, 0, 10240)) . "</pre></li>";
echo "</ul></li>";
} else if (stripos($overview->from, 'Mailer-Daemon@') === 0) { //catch all
echo "<li>{$overview->msgno}<ul>";
echo "<li>Sent: {$overview->date}</li><li>From: " . htmlspecialchars($overview->from) . "</li><li> Subject: {$overview->subject}</li>";
echo "<li><pre>" . htmlspecialchars(substr($body, 0, 10240)) . "</pre></li>";
echo "</ul></li>";
}
} //end foreach
echo "</ul>";
imap_close($mbox);
} //connected
ob_flush();
//NOW HAVE ARRAYS OF HARD AND SOFT FAILS, NEED TO PROCESS THEM…
echo "<p>Please note: 'Email address not found' can mean two things – 1) that you have a problem because a member record could not be found, or 2) that the member has already corrected the problem. If you see just a few messages like that, it usually indicates that the member already corrected the problem.</p>";
echo "</ul><h2>".count($softfails)." Soft Fails</h2><ul>";
asort($softfails);
foreach ($softfails as $fail){
list($email, $datetime) = explode(',', $fail);
echo "<li>$email – ";
ob_flush();
$member = \IPS\Member::load( $email, 'email' );
if( $member->member_id ) {
$member->bouncerSoftBounce( $datetime );
echo "set as soft bounce. ($datetime)";
} else {
echo "<b>EMAIL ADDRESS NOT FOUND</b>";
}
echo "</li>";
ob_flush();
}
echo "</ul>";
echo "<h2>".count($hardfails)." Hard Fails</h2><ul>";
asort($hardfails);
foreach ($hardfails as $fail){
list($email, $datetime) = explode(',', $fail);
echo "<li>$email – ";
ob_flush();
$member = \IPS\Member::load( $email, 'email' );
if( $member->member_id ) {
$member->bouncerHardBounce( $datetime );
echo "set as hard bounce. ($datetime)";
} else {
echo "<b>EMAIL ADDRESS NOT FOUND</b>";
}
echo "</li>";
}
echo "</ul>";
if ($_GET['lb'] != 'off'){
$sql = "SELECT `email` FROM `core_members` WHERE `temp_ban` = -1 OR `temp_ban` > NOW() + INTERVAL 8 WEEK";
$results = $bzDB->exec($sql); //REPLACE THIS WITH A SQL QUERY TO YOUR IP.BOARD DATABASE
echo "<h2>".count($results)." Long Bans (set as hard bounces)</h2><ul>";
foreach($results as $result){
$email = $result['email'];
echo "<li>$email – ";
ob_flush();
$member = \IPS\Member::load( $email, 'email' );
if( $member->member_id ) {
$member->bouncerHardBounce( time() );
echo "set as hard bounce.";
} else {
echo "<b>EMAIL ADDRESS NOT FOUND</b>";
}
echo "</li>";
}
}
echo "</body></html>";
?>
jay5r got a reaction from SeNioR- in Problem with `core_stream_subscriptions` causing huge increase in load
jay5r got a reaction from Square Wheels in Bouncing Emails
jay5r got a reaction from Maxxius in Bouncing Emails
jay5r reacted to stoo2000 in Bouncing Emails
jay5r got a reaction from bfarber in Bouncing Emails
