bernhara

playing around with my "robots.txt" configuration /admin/?app=core&module=promotion&controller=seo&tab=robotstxt, I can select any configuration: no robots.txt, default version, ... No effect. CloudFlare ignores my configuration. @Marc , is it better to open a ticket regarding this issue?

While making more tests with my robots.txt, I mentioned that my custom configuration is not always returned: often, only the CloudFlare part is return. And this behavior is random: sometimes, the content including my customization is returned, sometime not. Is there a cache issue?

However, my Google Search console tells me that Content-Signal is an unrecognized statement. Indeed, it’s ignored.

For various reasons, we need a custom configuration for robots.txt. After the upgrade to v5, I checked if something has changed for the SEO configuration. While doing this I discovered that CloudFlare catches the request for robots.txt and serves an altered version of robots.txt. https://www.planete-citroen.com/robots.txt A prefix is added which disallows all AI related search engines: # As a condition of accessing this website, you agree to abide by the following # content signals: # (a) If a Content-Signal = yes, you may collect content for the corresponding # use. # (b) If a Content-Signal = no, you may not collect content for the # corresponding use. # (...) # BEGIN Cloudflare Managed content User-agent: * Content-Signal: search=yes,ai-train=no,use=reference Allow: / # Training crawlers User-agent: Amazonbot Disallow: / User-agent: Applebot-Extended (...) # END Cloudflare Managed Content #----------------------------------------------- # BEGIN Customization # =================== (...) Since I'm not a crawling expert, I've no idea of the impact of these additional restrictions. We observed a small drop of our visibility on the Web, but I'm unable to know if this drop is related to the CloudFlare imposed rules. However, the CloudFlare rules do not correspond to what we want, since our aim is to share very largely and publicly the knowledge contained on our Topics. And AI learning and generation is part of this aim. Since managing the grants with robots.txt is not a very easy job, I have some difficulties to allow what CloudFlare disallows. REM: my GoogleSearch console says me that "Content-Signal" is not a valid statement.

Not exactly. The only thing we saw is that the daily revenues have dropped to almost zero. Before the situation became critical, we removed the AdSense code from our site, waiting for the end of the storm. And as expected, the bots probably understood that we are of no use to them. So, they stopped the attack and everything is OK again. Maybe CloudFlare helped to stop the attack. However, since we have no visibility on the CloudFlare activity, I can only speculate.

We are currently facing a huge attack: a large set a bots (with IP addresses located mainly in the US , Brazil, Columbia, ...) are opening connection to our server without getting the content of the pages. These geographical origins are not believable for our site, since we are a French Forum with mostly all users located in France. Many of these connections are counted by IPS as "visitors". Indeed, all our counter are becoming completely wrong and does not represent anymore the real usage of our Forum. However, this is not really a problem, since we are on the Invision Cloud solution, which has all the needed protections to maintain our Forum 100% up and responsive. Our problem lies on the GoogleAd side, which gives us a significant part of our revenues (we push Ads to our guest and free user). The current attack makes the GoogleAd system become crazy: when a bot gets the page, the GoogleAd code is pushed to the bot, but no revenue is generated by Google, since the bot does not behave in the expected way. As a consequence our Ad revenue have dramatically dropped and we have been tagged by Google as an untrusted site. Regarding this problem, I have 2 questions: is someone else facing the same problem? as we are hosted on a Cloud instance managed by Invision, we have no access to the incoming network filtering configuration. Is there a way to make a configuration at IPS Admin level that would reject all the ill traffic or make the GoogleAd script resistant to this kind of attack?

Hi; I have a very simple question which may appear stupid. Forgive me if so🤪 We are still on a V4 instance and I started to use the "Commerce" application which fulfills exactly our requirement (management of a yearly Club subscription). It responds exactly to the question I had a long time ago: https://invisioncommunity.com/forums/topic/478373-managing-members-with-yearly-subscription Before relying to much on the Commerce application, my question relates to the future if this application. Will this application be provided and supported in the long term future?