Hi all,

Looking for anyone else seeing this, or who has already solved it.

The problem: Google Search Console shows severe indexing failure on our Community in the Cloud-hosted forum, bullnoseforum.com. Google knows about roughly 39,000 URLs, but currently only 2 are indexed. Crawl Stats shows about 84% of Googlebot requests over the last 90 days failing with “Other client error (4XX).”

This is not new. Invision Support has confirmed that automated traffic was temporarily blocked during platform instability in late August, and our Search Console data shows very large failure spikes in late August and again around September 1–4, with failures continuing through at least September 15.

What we have ruled out on our end: Our own Cloudflare account is DNS-only, not proxied. Bot Fight Mode is off, there are no WAF or rate-limiting rules, and there are no Security Events because our Cloudflare zone is not in the HTTP request path.

In Invision AdminCP, robots.txt does not block /forum/ or /topic/; there are no per-page noindex overrides; live pages do not return an X-Robots-Tag; there are no banned IP addresses; and the login, posting, and registration protections do not apply to ordinary anonymous page requests.

What we found directly: A third-party fetch/render test against one of the affected URLs returned HTTP 403 with cf-mitigated: challenge and Cloudflare’s “Just a moment...” challenge page. Since our own Cloudflare zone is DNS-only, that challenge is coming from Cloudflare in front of Invision’s hosted infrastructure.

I realize that test is not Googlebot itself, so by itself it does not prove Googlebot is being challenged. The unresolved issue is that Google Search Console itself records real Googlebot requests to ordinary public forum and topic URLs receiving 4XX responses.

One example:

September 15, 2026, 3:26 PM PT
https://bullnoseforum.com/forum/23-for-sale-trade-or-free-to-a-good-home/

Search Console records that Googlebot request as “Other client error (4XX).”

What Support has said: We were initially told that automated traffic had been blocked during the August platform instability and Google may have been caught temporarily. We were later told that AI bots are blocked but Google is exempt. Most recently, Support stated that “we do not block Google search bots to any degree on our platform.”

That leaves us with a contradiction we have not been able to resolve: Google says Googlebot is receiving 4XX responses, while Invision says Googlebot is exempt from its bot blocking.

There is also a recent Cloudflare change that may be relevant to failures from September 15 onward, although it cannot explain the August failures. On September 15, Cloudflare changed its handling of mixed-use crawlers such as Googlebot. A Training setting of Block now blocks those crawlers for search as well, while the new Disallow AI Training setting is intended to permit search crawling while refusing AI training.

Has anyone else using Community in the Cloud checked Google Search Console Crawl Stats and seen a similar pattern of 4XX failures? If so, did you determine what was generating them? And have you gotten it fixed? We are essentially non-existent to Google.

Thanks,

Gary

Search crawling is indeed allowed. Their AI bot is indeed the AI training. The examples shown in your ticket show this indeed to be AI bots. Please respond to your ticket if you believe this is not the case and I can certainly pass that on

Hi Marc,

Yes, please pass this on.

I agree that the TameTheBots test Chris ran may have been classified as an AI bot. We used that only to demonstrate that Invision's Cloudflare layer is returning a 403 challenge to some automated traffic.

But that is separate from the original issue.

The 171 examples I supplied came directly from Google Search Console's Crawl Stats report for bullnoseforum.com. Google identifies those as Googlebot crawl requests, and reports that they received "Other client error (4XX)" responses.

Cloudflare's current documentation also distinguishes crawler behavior by purpose and notes that some crawlers may have more than one behavior, including Search and Training. So an "AI bot" classification in Cloudflare does not by itself establish that the request was unrelated to legitimate search crawling.

So the question is not whether the TameTheBots request was an AI bot. The question is why Google's own Search Console records legitimate Googlebot requests receiving 4XX responses from the forum.

Please do pass this on for further investigation.

Thanks,

Gary

3 hours ago, Marc said:

Please respond to your ticket

Sorry, I missed that. But I have now responded to the ticket as well.

For all that may be watching this, Marc is investigating in the background, as am I. And I found one more data point that may be worth other Invision Cloud customers checking, which I've shared with him.

Our Google Search Console Performance graph shows a very abrupt drop in Google impressions and clicks around May 4–6, months before the August bot-mitigation incident we've been discussing. Impressions went from roughly 2,000/day to almost nothing and have never recovered.

Search Console shows no Manual Actions and no Security Issues.

I'm attaching the graph. If anyone else here runs an Invision-hosted community, it might be useful to look at your 12-month Search Console Performance graph and see whether anything similar happened around early May.

I'm not saying this was an Invision-wide event. I'm trying to determine whether the timing is unique to our community or shared by others.