SeNioR- Posted January 9, 2023 Posted January 9, 2023 I suggest excluding pages like /modcp/ from IndexNow as the crawler can't access them anyway. 207.46.13.11 - [07/Jan/2023 +0100] "GET /modcp/reports/4120/ HTTP/1.1" 403 11672 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
Daniel F Posted January 9, 2023 Posted January 9, 2023 IPS won't send any requests to IndexNow suggesting to index such an URL. I guess you mean to add an no index directory to the robots.txt file?
SeNioR- Posted January 9, 2023 Author Posted January 9, 2023 (edited) ok thanks, Bing Search shows me that this page was sent by IndexNow. Don't know why, maybe a single case. 55 minutes ago, Daniel F said: I guess you mean to add an no index directory to the robots.txt file? no, not that 🙂 Edited January 9, 2023 by SeNioR-
aia Posted January 9, 2023 Posted January 9, 2023 (edited) @SeNioR- check the value of this setting in Cloudflare: Edited January 9, 2023 by 13. SeNioR- 1
Daniel F Posted January 10, 2023 Posted January 10, 2023 18 hours ago, 13. said: @SeNioR- check the value of this setting in Cloudflare: Wouldn't this show Cloudflare as the source? At least that's what I get when the crawler hints are enabled.
SeNioR- Posted January 10, 2023 Author Posted January 10, 2023 I had two options enabled, I didn't notice it 😛
Recommended Posts