AutoItScript Posted September 1, 2015 Posted September 1, 2015 (edited) Having been under heavy spam attack recently the plugin I was previously using had a few bugs and I needed some more options so this is the result. Features Performs various checks during user sign-up to prevent spammers Checks the Stop Forum Spam service and uses the "confidence" part of the API to reduce false positives Checks the Project Honeypot service with a configurable threat level (API key required) Checks one or more standard DNS Block Lists (DNSBL) such as xbl.spamhaus.org Checks a built-in list of hundreds of disposable email providers so that you don't need to add separate Ban Filters in the ACP Logs actions to the built-in IPS Spam Service Logs (different response codes are used for Stop Forum Spam, PH, Email so you can see which service triggered an action) This is what I use on my own forum, so suggestions for improvements are welcome. The disposable email list is just a built in list of a few hundred hosts I found at the moment. I'm not sure the best way of making this list configurable/updated. I found some services on the web but they aren't free and I'm not really interested in non-free services for this plugin unless there is demand. Edited September 1, 2015 by AutoItScript
AutoItScript Posted September 2, 2015 Author Posted September 2, 2015 I'm still getting a fair bit of Korean spam which is being created by users who aren't on any of IPS/StopForumSpam/http:BL lists. I'm testing a feature where a regular expression can be applied to post titles and content. If it matches the post is automatically flagged for moderator approval. This means that the forum will still look clean for users and moderators can then clean it up easily with the Flag As Spammer tool. I'm testing the changes on my forum at the moment - I'll update the plugin on Marketplace after testing looks ok. LaCollision 1
GriefCode Posted September 2, 2015 Posted September 2, 2015 Generally sounds like a great plugin, once i got time i will look into it. And i like that you told the truth about the remaining spam, i think thats pretty interesting how you want to achieve this with a regex. Could you give more information how you plan to integrate it? Is it some kind of irony that your forum name is 'Spam Happy'?
AutoItScript Posted September 2, 2015 Author Posted September 2, 2015 Spam Happy is an automatic IPS rank I think, I've not got around to changing it yet No point not being honest about the functionality - I'm not trying to sell anything. I'm not interested at all in writing plugins/code for IPS 4 but I'm just finding it a necessity at the moment to make things work how I/my community want. And I'm geeky enough to enjoy the challenge so it's not all bad We've had up to 40 of these korean spammers per day each creating up to 10 topics each. It looks terrible until the moderators clean up. So I'm trying all kinds of things to balance ease-of-registration but preventing spam. Here's a screenshot of the new feature as it stands: The php regex I'm testing at the moment is '~\p{Hangul}|\p{Han}|\p{Cyrillic}~u' which flags any Russian/Chinese/Korean. At that point the user has already registered so I'm then flagging the matching topic/post and switching it to moderator approval and the moderators receive a notification. So users never even see it. If the post is legitimate then it can just be approved. Otherwise I'm clicking Flag As Spammer which deletes all the content and bans the users (that's how I have it setup). Here is a screenshot of my forum right now after I've had it running for a while. LaCollision and GriefCode 2
GriefCode Posted September 2, 2015 Posted September 2, 2015 Okay, thats pretty interesting. You only need to maek sure to people who actually have this languages, that they dont use the plugin or disable at least the regex I like free plugins, because i like to browser through things and how they are solved. Im not interested into using them How does it come that the IPB Spam service isnt gripping into that? I'm not sure, but that kind of spam should be detected, a new member with 4 posts+ within 7 minutes. Thats basically always spam and should be at least flagged.However in the screenshots you could also simply match the display name Well i thought its the automated name, but the irony in that is still funny Nice work, keep it up
AutoItScript Posted September 2, 2015 Author Posted September 2, 2015 The IPS service (or StopForumSpam/http:bl) seems to be missing 99% of these spammers. If you do a google for korean spammers you'll see that they've been a plague this year with many big forums have 10/20 pages of spam on the front pages. They seems to be highliy sophisticated and using new accounts/IPs all the time. When I click Flag As Spammer it will tell the IPS service about this particular email address so someone on a less busy board may benefit. GriefCode 1
AutoItScript Posted September 3, 2015 Author Posted September 3, 2015 Finished testing the new version and I've had my first Korean-spam free day today v1.0.2 uploaded to marketplace. Added option to check IP addresses during registration and take action based on the country of origin Added option to require moderator approval for posts and topics based on a regular expression check of the content. Users with already approved content can be excluded. In addition to deny/allow/admin approve registrations you can also flag the new member to require moderator approval for new content
AutoItScript Posted September 7, 2015 Author Posted September 7, 2015 Added option to check posts with the Akismet service and auto-moderate if matchedI've got visible spam down to 0 this week on my forum. Down from about a maximum of 90 posts a day before I started on the plugin LaCollision and axpv 2
PANL Posted September 8, 2015 Posted September 8, 2015 So when you have this country check. what will the user see that registrate from a country that's not allowed?Will they get a error message with some explanation? that will be userfriendly if GEO has it wrong, or someone with your language access the forum from a other country. Nice tool by the way
AutoItScript Posted September 8, 2015 Author Posted September 8, 2015 It depends on the action you set:- Deny: They get the same message as the IPS Spam Service which is something like "You are not allowed to register" with a link to the Contact Us form.- Allow with admin approval: They get a message saying that they will receive an email message to validate once admin has approved the account- Moderate: They register as normal, but when they go to post they will be in the moderator queue.I've actually set my forum up so that it just lets them register anyway but they are placed on moderation. Less trouble for the user (if they are a real user) than getting totally blocked from registration and still easy to clearup or see from the first post if they are a spammer.
PANL Posted September 8, 2015 Posted September 8, 2015 (edited) It depends on the action you set: - Deny: They get the same message as the IPS Spam Service which is something like "You are not allowed to register" with a link to the Contact Us form. - Allow with admin approval: They get a message saying that they will receive an email message to validate once admin has approved the account - Moderate: They register as normal, but when they go to post they will be in the moderator queue. I've actually set my forum up so that it just lets them register anyway but they are placed on moderation. Less trouble for the user (if they are a real user) than getting totally blocked from registration and still easy to clearup or see from the first post if they are a spammer. aaah great! i just test the disposable but 10 minute mail still works. http://10minutemail.com/10MinuteMail/index.html - g10239209@trbvm.com http://www.my10minutemail.com - zv1oh58nofeipu44@my10minutemail.com Is there maybe a way to add more domains? Edited September 8, 2015 by PANL
icedream Posted September 21, 2015 Posted September 21, 2015 Great mod, however I have to disable the "disposable email address list" feature....it lists qq.com as disposable email address, but qq.com is the most popular email address in China. Anyway, I find disposable-email-domains, I guess this might be helpful to you, good news is that qq.com is not listed. :)
AutoItScript Posted September 21, 2015 Author Posted September 21, 2015 Thanks. I'll update with that list next time I do a release
gbarry Posted December 22, 2015 Posted December 22, 2015 Appreciate the work put into this--while I know it's something you built for your own site, I was wondering if you've tested recently with the latest version of IPS4. Particularly interested in the keyword blocker. Ran a few test posts with a few expressions (including your language blocks) and the offending post generates an EX2 error code. Funny thing is, the post still makes it through unmoderated, but does not trigger a notification email.
AutoItScript Posted January 8, 2016 Author Posted January 8, 2016 I've not had chance to check it with the latest version yet - I need to get my site updated as well. I'll make a note to recheck everything.
AutoItScript Posted January 9, 2016 Author Posted January 9, 2016 (edited) On 22 December 2015 at 6:16 AM, gbarry said: Appreciate the work put into this--while I know it's something you built for your own site, I was wondering if you've tested recently with the latest version of IPS4. Particularly interested in the keyword blocker. Ran a few test posts with a few expressions (including your language blocks) and the offending post generates an EX2 error code. Funny thing is, the post still makes it through unmoderated, but does not trigger a notification email. I just upgraded my dev site to 4.1.6.1 and as far as I can tell everything is looking ok. I looked at the code changes and there doesn't seem to be anything that should have changed. Is anyone else able to replicate this? Edit: Nah, something is a bit screwy, I'll have to investigate more. Edit2: Actually the bug I saw was nothing to do with the plugin, some sort of advanced search error that happens even without the plugin. Edited January 9, 2016 by AutoItScript
AutoItScript Posted January 14, 2016 Author Posted January 14, 2016 I've reworked the plugin (the auto moderation parts) based on some new API features in 4.1.6 -it makes the code much neater and hopefully fixes any problems. I'm upgrading my live site this weekend and if all goes well I'll release the new version then. It will require 4.1.6+ though. Martin1980 and gbarry 2
AutoItScript Posted January 18, 2016 Author Posted January 18, 2016 I've not uploaded the new version yet, currently testing it on my 4.1.7 live site and it's looking ok. Sam A 1
Teddy Rogers Posted January 20, 2016 Posted January 20, 2016 @AutoItScript, I noticed your plugin filters post content. Is it possible for it to filter users posting attachments so they can be checked and moderated for approval? Ted.
AutoItScript Posted January 20, 2016 Author Posted January 20, 2016 Possibly. I've no idea how attachments work but I can have a look if the relevant bits are exposed when my code runs.
Teddy Rogers Posted January 21, 2016 Posted January 21, 2016 Would be greatly appreciated, thank you for taking a look... Ted.
LaCollision Posted January 21, 2016 Posted January 21, 2016 Hi, Can't download version 1.0.3… when I try the change log version with 1.0.3, this is the 1.0.4 I get. Could it be an IPS bug? Thanks
LaCollision Posted January 22, 2016 Posted January 22, 2016 Hi, Outstanding plugin, THANK YOU! Would it be possible, for the regex verification, to be able to mark as a spammer, instead of putting the content under moderation review? That would be amazing! Thanks again, AutoItScript 1
Recommended Posts