Hey IPB Team!

Community Mailings are by far one of the most important methods to keep your visitors visiting^^

After several thousand members it becomes quite hard to handle all the junk you get back from those invalid mail addresses and manually sort out the members that are no longer with us.

I am using phplist, a very fast and good tool for mailings. It has a bounce function built it. E-mail is sent out with an 'envelope-to' option that redirects mailer-daemon errors to a special mailbox. in my case bounces@mydomain.com. Normal replys will be sent to your email-account as usual.
Then i can click 'process bounces' and phplist imports all the mails from the bounces@domain.com mailbox and after a predefined number of bounces (lets say five) it automatically unsubscribes the user from our mailinglist.

This way we

a) always know how many people we really reach
b) don't have to clean up our private mailboxes with THOUSANDS of mailer-daemon-mails
c) have more time to watch our favorite tv show

:)

What do you think?

Cheers
-Daniel

I think the functionality is great, but unlikely to be included with IPB (difficult for us to support with so many potential server setups).

I think the functionality is great, but unlikely to be included with IPB (difficult for us to support with so many potential server setups).

Hey Brandon!

Why is this a problem in terms of the server setup? The entire bounce handling is being done with a pop3 account. Those are always the same. Phplist imports all emails from this pop account to a mysql database and processes the bouces there. There's a specific set of rules that identify mailer-daemon mails. :)

function processBounce ($link,$num,$header) {

 global $tables;

 $headerinfo = imap_headerinfo($link,$num);


 $body= imap_body ($link,$num);

 $msgid = 0;$user = 0;

 preg_match ("/X-MessageId: (.*)/i",$body,$match);

 if (is_array($match) && isset($match[1]))

	$msgid= trim($match[1]);

 if (!$msgid) {

	# older versions use X-Message

	preg_match ("/X-Message: (.*)/i",$body,$match);

	if (is_array($match) && isset($match[1]))

 	$msgid= trim($match[1]);

 }


 preg_match ("/X-ListMember: (.*)/i",$body,$match);

 if (is_array($match) && isset($match[1]))

	$user = trim($match[1]);

 if (!$user) {

	# older version use X-User

	preg_match ("/X-User: (.*)/i",$body,$match);

	if (is_array($match) && isset($match[1]))

 	$user = trim($match[1]);

 }


 # some versions used the email to identify the users, some the userid and others the uniqid

 # use backward compatible way to find user

 if (preg_match ("/.*@.*/i",$user,$match)) {

	$userid_req = Sql_Fetch_Row_Query("select id from {$tables["user"]} where email = \"$user\"");

	if (VERBOSE)

 	output("UID".$userid_req[0]." MSGID".$msgid);

	$userid = $userid_req[0];

 } elseif (preg_match("/^\d$/",$user)) {

	$userid = $user;

	if (VERBOSE)

 	output( "UID".$userid." MSGID".$msgid);

 } elseif ($user) {

	$userid_req = Sql_Fetch_Row_Query("select id from {$tables["user"]} where uniqid = \"$user\"");

	if (VERBOSE)

 	output( "UID".$userid_req[0]." MSGID".$msgid);

	$userid = $userid_req[0];

 } else {

	$userid = '';

 }

 Sql_Query(sprintf('insert into %s (date,header,data)

	values("%s","%s","%s")',

	$tables["bounce"],

	date("Y-m-d H:i",@strtotime($headerinfo->date)),

	addslashes($header),

	addslashes($body)));


 $bounceid = Sql_Insert_id();

 if ($msgid == "systemmessage" && $userid) {

	Sql_Query(sprintf('update %s

 	set status = "bounced system message",

 	comment = "%s marked unconfirmed"

 	where id = %d',

 	$tables["bounce"],

 	$userid,$bounceid));

 	logEvent("$userid ".$GLOBALS['I18N']->get("system message bounced, user marked unconfirmed"));

 	addUserHistory($user,$GLOBALS['I18N']->get("Bounced system message"),"

	<br/>".$GLOBALS['I18N']->get("User marked unconfirmed")."

	<br/><a href=\"./?page=bounce&id=$bounceid\">".$GLOBALS['I18N']->get("View Bounce")."</a>


	");

	Sql_Query(sprintf('update %s

 	set confirmed = 0

 	where id = %d',

 	$tables["user"],

 	$userid));

 } elseif ($msgid && $userid) {

	Sql_Query(sprintf('update %s

 	set status = "bounced list message %d",

 	comment = "%s bouncecount increased"

 	where id = %d',

 	$tables["bounce"],

 	$msgid,

 	$userid,$bounceid));

	Sql_Query(sprintf('update %s

 	set bouncecount = bouncecount + 1

 	where id = %d',

 	$tables["message"],

 	$msgid));

	Sql_Query(sprintf('update %s

 	set bouncecount = bouncecount + 1

 	where id = %d',

 	$tables["user"],

 	$userid));

	Sql_Query(sprintf('insert into %s

 	set user = %d, message = %d, bounce = %d',

 	$tables["user_message_bounce"],

 	$userid,$msgid,$bounceid));

 } elseif ($userid) {

	Sql_Query(sprintf('update %s

 	set status = "bounced unidentified message",

 	comment = "%s bouncecount increased"

 	where id = %d',

 	$tables["bounce"],

 	$userid,$bounceid));

	Sql_Query(sprintf('update %s

 	set bouncecount = bouncecount + 1

 	where id = %d',

 	$tables["user"],

 	$userid));

 } elseif ($msgid === 'systemmessage') {

	Sql_Query(sprintf('update %s

 	set status = "bounced system message",

 	comment = "unknown user"

 	where id = %d',

 	$tables["bounce"],

 	$bounceid));

 	logEvent("$userid ".$GLOBALS['I18N']->get("system message bounced, but unknown user"));

 } elseif ($msgid) {

	Sql_Query(sprintf('update %s

 	set status = "bounced list message %d",

 	comment = "unknown user"

 	where id = %d',

 	$tables["bounce"],

 	$msgid,

 	$bounceid));

	Sql_Query(sprintf('update %s

 	set bouncecount = bouncecount + 1

 	where id = %d',

 	$tables["message"],

 	$msgid));

 } else {

	Sql_Query(sprintf('update %s

 	set status = "unidentified bounce",

 	comment = "not processed"

 	where id = %d',

 	$tables["bounce"],

 	$bounceid));

 	return false;

 }

 return true;

}

Would it be possible to add the message envelope option for starters so that i am not being bothered by all those mailer-daemon mails each time i send out a newsletter?

Line 3 "imap_headerinfo". What if this function isn't available?

"X-MessageId" vs "X-Message". What if the mail server doesn't send or set either of these?

We make mass-deployable software, and in doing so have to consider *everyone's* server setup, and whether the feature can work on other server setups, or whether it's acceptable to list such a feature as "requires x, y or z". Stuff like this tends to be hard to get working across everyone's server setup is all I was saying.

imap_headerinfo is available since php4 - X-Message tags etc. aren't necessary as well - phplist works well with other setups too. I'd also like to note, that if one out of thousand error responses do not get recognized it isn't a major problem while having literally thousands of mailer-daemon mails every WEEK in my inbox very much is. ;)

www.phplist.com offers a free download of PHPlist. It's being used by major comapnies and really works well. Please take a look at their functions in terms of bounce handling - this software has saved me hours if not weeks of work :)

imap_headerinfo is available since php4 - X-Message tags etc. aren't necessary as well - phplist works well with other setups too. I'd also like to note, that if one out of thousand error responses do not get recognized it isn't a major problem while having literally thousands of mailer-daemon mails every WEEK in my inbox very much is. ;)

www.phplist.com offers a free download of PHPlist. It's being used by major comapnies and really works well. Please take a look at their functions in terms of bounce handling - this software has saved me hours if not weeks of work :)

We make forum software though. I would expect mailing list software that is coded to handle mailing lists specifically, would have functionality to handle those mailing lists (including bounces). You can't expect IPB to have every feature possibly available however. ;)

The function imap_headerinfo, while available since PHP 4, requires the IMAP extension to be installed.

http://www.php.net/manual/en/imap.installation.php

Again, there's no reason to guarantee this function is installed at present.



I'm not discounting the idea itself (it would be nice to shut off emails for users whose addresses are bouncing), but I don't think this sort of functionality is reliable enough in the type of package we distribute. That's just a prediction.

1) You can easily create a routing change that says any mail that is from <> (which is the normal return path on an NDR) goes to xx@thismailbox.com
2) PHPList is also not all that great".. I've dealt with a good 3-4 dozen companies who don't have it utilized properly and only create 20-30K Back Scatters a week and they get black listed

In reality all the configuration ask you're wanting is to import everything into a database, you can already do that with Squirrel, Horde or Cube once you learn what fields to export you can CRON it to to an Update on your IPB DB on a condition where email equals xxx allow email equals 0

1) You can easily create a routing change that says any mail that is from <> (which is the normal return path on an NDR) goes to xx@thismailbox.com

Right. One could. I want to listen in on the phone calls where tech support have to explain this to the scrapbooker soccer mom running a forum for her girl scout troop, however. :lol:

You have to keep in mind, "easily" is an extremely relative term on the internet. We have many customers who are not capable of installing IPB themself because they don't know what an FTP is or where to get one.

Right. One could. I want to listen in on the phone calls where tech support have to explain this to the scrapbooker soccer mom running a forum for her girl scout troop, however. :lol:

You have to keep in mind, "easily" is an extremely relative term on the internet. We have many customers who are not capable of installing IPB themself because they don't know what an FTP is or where to get one.

Good point, I think I just had some cold shivers down my spine.. I do feel your pain when one of our Enterprise customers goes "I don't know what a smart host is" and they're the bloody messaging admin..

Right. One could. I want to listen in on the phone calls where tech support have to explain this to the scrapbooker soccer mom running a forum for her girl scout troop, however. :lol:

oh come on, wouldn't that be adorable? ;)

I get your point regarding the complexity of this feature. So what about that envelope-to field? This would work with EVERY Setup, doesn't require much more than a soccer mom to fill out AND would provide an easy way to route Bounces to a different mailbox without the needed knowledge to route this by yourself.

:)

This is a great idea, maybe not for IPS, but for a MOD feature request

We sent 70K emails and really need to skip those emails that were bounced back before

Or you could be using those to clean up your mailing lists.. Sending any quantity of mail that bounces more than once is a bad thing..

If you really don't want to do that, than just setup a rule to black hole mail with a <> return path (which is what an NDR SHOULD have)

Or you could be using those to clean up your mailing lists.. Sending any quantity of mail that bounces more than once is a bad thing..

If you really don't want to do that, than just setup a rule to black hole mail with a <> return path (which is what an NDR SHOULD have)

could you elaborate on this please?

I definetely sybscribe to the ideea of reading or collecting bounced mails.
I have the same problem: 32000 members.. 3-4000 bounces.

It would be absolutely great to have a way to read the bounced mails and put those members in some other group, delete them, or delete them with a condition (to have less than n posts)

I am currently running a dedicated mailing software, that reads bounces, collects them and filters the database, but only for the mails sent through that software, it cannot read any other bounces from the forum, because it embedds some sort of code in the email that when is bounced it can read it easily (this was their explanation when i tried to ask them about reading other bounce messages sent from my forum):

And here i am: The forum is still sending email messages as notifications:
- birthday greeters
- inactivity emails, to try and bring back the members on the forum
- and others...