Spam Prevention Question

[Arthmoor]

For the last 2 days, our site has gone from essentially zero spam to suddenly being under heavy assault by bots all registering to post airfare scamvertising.

Last night I took advice from another thread suggesting switching the registration captcha to hCaptcha instead of the default. That hasn't even slowed them down one bit, because again today I've had to flag 2 dozen accounts (and my moderators are fielding dozens more I haven't seen myself).

Every single one of the accounts we've all flagged in the last 2 days have bypassed the spam defense system with a score of 1, so it isn't an issue of being too generous with account registrations. They must also be bypassing the question & answer challenge we have set up in addition to the standard captcha. Further SOME of the posts are being caught and flagged for manual approval, but not all of them, despite them all literally using the same colorful rainbow emojis in their titles and text and all linking to the same set of scam websites for airline tickets.

The other thread suggested turning off all of the social logins, like Facebook, Google, and Twitter, but for us those were never activated to begin with. Only the standard IPB login system is enabled.

So in light of this I have a couple of questions:

1. When a spammer is flagged, how long does it take for the defense system to learn this and act accordingly to stop them from registering?

2. When a clearly scammy topic is posted, why does the forum only catch some of them and not others?

3. Is there a way to look into the registration data and see which question and answer set someone had to go through? Maybe it's simply a case that the botnet running this attack has found one or more then can readily answer and we just need to change or remove them.

4. WHY oh WHY does the "Contact Us" form not use any form of captcha on it by default?!? (this is a whole other source of problems not related to this)

[Marc]

9 hours ago, Arthmoor said:

1. When a spammer is flagged, how long does it take for the defense system to learn this and act accordingly to stop them from registering?

We are unable to give specific information on how the spam service works internally unfortunately

9 hours ago, Arthmoor said:

2. When a clearly scammy topic is posted, why does the forum only catch some of them and not others?

Spam detection is simply not an exact science unfortunately. There will be times where these people who are signing up are actually real people creating the accounts by hand. In that case, they will indeed get through. And at times you will get an influx of those as you seem to be getting currently

9 hours ago, Arthmoor said:

3. Is there a way to look into the registration data and see which question and answer set someone had to go through? Maybe it's simply a case that the botnet running this attack has found one or more then can readily answer and we just need to change or remove them.

There is no way in which to do this at the present time. Its usually items which are simply answered by google they will get through with no problem at all if they are bots. For example on yours

 

If google is getting the answer right like that, then so will the bot.

9 hours ago, Arthmoor said:

4. WHY oh WHY does the "Contact Us" form not use any form of captcha on it by default?!? (this is a whole other source of problems not related to this)

Because often the reason people will contact on a contact form is to tell you they are having issues registering. If that issue is they cannot register due to the spam protection you have in place, you leave them in limbo

 

Looking at your spam defense setup there, the first thing that is glaring here is that you have changed the default setup on the spam defence tab

As you can see there, you are actually allowing users to register as normal whn they are likely to be spam. The default for 3/4 would usually be manual review on a default setup. 

10 hours ago, Arthmoor said:

Last night I took advice from another thread suggesting switching the registration captcha to hCaptcha instead of the default. That hasn't even slowed them down one bit, because again today I've had to flag 2 dozen accounts (and my moderators are fielding dozens more I haven't seen myself).

Have you taken a look at your settings on hcaptcha, as you can set higher levels on this if you are still seeing issues. However do bare in mind what I mention above. These may be actual people doing the spamming. If they are, then of course they will get through these.

 

[Arthmoor]

I wasn't asking you how the filtering works, I just wanted to know how long it takes for reported spam to get incorporated into the detection.

So I'll ask a related question instead. Once we flag a spammer, do we need to leave their data in the system or can those accounts and content be permanently removed without issue?

With the registration data, can I suggest that a feature be added at some point so that we can see which questions the spammers have been asked? It's been my experience elsewhere that when they do break this, they tend to only break a specific question or two and replacing them goes a long way toward tripping them up. But we can't do that right now because the system doesn't keep track of it.

Regarding the spam defense setup, perhaps you missed where I told you that every single one of these spammers scored a 1, so changing the level 3 action won't matter. Besides, we've had legitimate users get dinged at level 3 and require manual approval far too often so it seems to me there's something very wrong there when the only people being inconvenienced in some way are the legit users.

Yes, I did look at the hCaptcha settings and bumped it up to moderate difficulty. I guess since nothing the system you've provided us is actually doing anything, I can try bumping it up to the max difficulty.

[Randy Calvert]

You can delete it...  nothing "breaks" if you remove the account.  But that means that email or username could possibly be used again.  

Personally I use CleanTalk to help reduce spam on my site.  It blocks a LOT of bad stuff.  It's not perfect, but it stops 80-90% of it.  It's $12 a year, so not a high cost at all and does a pretty good job.

 

[Jim M]

23 minutes ago, Arthmoor said:

I wasn't asking you how the filtering works, I just wanted to know how long it takes for reported spam to get incorporated into the detection.

So I'll ask a related question instead. Once we flag a spammer, do we need to leave their data in the system or can those accounts and content be permanently removed without issue?

With the registration data, can I suggest that a feature be added at some point so that we can see which questions the spammers have been asked? It's been my experience elsewhere that when they do break this, they tend to only break a specific question or two and replacing them goes a long way toward tripping them up. But we can't do that right now because the system doesn't keep track of it.

Regarding the spam defense setup, perhaps you missed where I told you that every single one of these spammers scored a 1, so changing the level 3 action won't matter. Besides, we've had legitimate users get dinged at level 3 and require manual approval far too often so it seems to me there's something very wrong there when the only people being inconvenienced in some way are the legit users.

Yes, I did look at the hCaptcha settings and bumped it up to moderate difficulty. I guess since nothing the system you've provided us is actually doing anything, I can try bumping it up to the max difficulty.

Our Spam Defense system utilizes data points to signify a spammer. There really isn't any information I can reveal about this other than the best advice is to mark these spammers as such as the more data points we have, the more quickly they will be blocked by the system so it is in the best interest of all administrators to flag spammers that come to their community.

As Randy mentioned, you can remove the account if you wish but they could likely turn around in the near future to register again.

[Arthmoor]

The reason I asked about deleting the accounts wasn't about whether they could come back and register again. Your system is utterly failing to do that anyway.

A new frustration in all this has come up though. The realization that the post filtering feature only checks the content of the post body. It doesn't check the titles at all. So despite the fact that I added block terms to the filter, I logged on today to yet more spam from these people with the blocked terms in the title but not the post body. That doesn't really do any good when it's the titles they're using to the greatest effect. Including their stupid little phone emojis. So I don't get why topic titles aren't being checked by this. That seems like a pretty big oversight.

And I gotta say, it was a delicious irony to find the emails coming from this site to tell me that there were new posts in this thread are being flagged as spam by Gmail.

All in all, I gotta say that I'm not really happy with the poor showing that the spam defense system is presenting. Seems pretty damning to me when other customers are recommending Marketplace packages that do a better job of this than a full team of professionals.

[Randy Calvert]

The plugins that are recommended are done by folks that live/breath spam mitigation across forums, Wordpress sites, etc. 

IPS spam filtering system is a side project that is done in conjunction with the full suite development. 

So I think you’re thinking about this backwards… where some of the third parties can do more because that’s all they’re doing instead of doing it as part of dozens of other projects and where it’s not something that is DIRECTLY revenue generating. 

[Arthmoor]

For the cost of the license we should be getting that level of spam protection as part of the package, not as part of a 3rd party purchase on top of the package.

IMO, whatever system is currently in use is worthless. Maybe it's time for IPB to partner up with Akismet and implement something centered around that, because that actually works and correctly learns from behavior immediately rather than waiting some unspecified super secret amount of time before obvious as hell spam is blocked by the system.

[Randy Calvert]

Given this is not a break/fix issue… if you want to make a suggestion, post it in the Features Suggestions forum. Otherwise it will be lost in the sea of technical support requests. 

[Marc]

Please post any suggestions you would like to see in the future within our feature suggestions forum. There are some suggestions you have there which will just be lost within a support ticket otherwise 🙂 

On 12/23/2022 at 7:32 PM, Arthmoor said:

A new frustration in all this has come up though. The realization that the post filtering feature only checks the content of the post body. It doesn't check the titles at all. So despite the fact that I added block terms to the filter, I logged on today to yet more spam from these people with the blocked terms in the title but not the post body. That doesn't really do any good when it's the titles they're using to the greatest effect. Including their stupid little phone emojis. So I don't get why topic titles aren't being checked by this. That seems like a pretty big oversight.

It has to be said here, that you seem to have missed this may simply be a bug. Do you have an example we can take a look at on your site?

[Arthmoor]

On 12/26/2022 at 5:13 PM, Marc Stridgen said:

It has to be said here, that you seem to have missed this may simply be a bug. Do you have an example we can take a look at on your site?

I don't, because who wants to keep spam visible on the site after all?

It's easy to reproduce though. Set up a post filter to block a word, making sure that even admins are subject to the filter. Put the offending word into the body of a post. It will block submission informing you that you included the forbidden word. Now try it again, only put it in the title of the post instead. The system will allow it to go through. This should definitely not be happening since spammers rely mostly on the titles to attract attention from search bots.

[Arthmoor]

One of my moderators captured this as an example. In this pic, both "Airlines" and "Reservation Number" are listed as blocked in the word filters, but you can see they ended up in the title all the same.

I've also specified that red phone emoji as well but that's not working either.

You'll notice the blocked terms are nowhere in the post body at all, because the system stopped them from posting it with those terms in it.

[Marc]

I have just taken a look at this, and it is indeed correct. Word filters, at present, work with only the body of the text. I will add a note to bring this up internally, however this is working as intended currently

[Marc]

Thank you for bringing this issue to our attention! I can confirm this should be further reviewed and I have logged an internal bug report for our development team to investigate and address as necessary, in a future maintenance release.

 

[loccom]

we are using clean talk app and it works great... we have got rid of 1000 spam registrations in 2 weeks

Edited January 16, 2023 by loccom

[Marc]

On 1/3/2023 at 8:15 AM, Marc Stridgen said:

I have just taken a look at this, and it is indeed correct. Word filters, at present, work with only the body of the text. I will add a note to bring this up internally, however this is working as intended currently

This issue was resolved in the recently released 4.7.13 release. Please update in order to fix this issue, and if you see any problem after that point, please let us know.