Jump to content

Excluding some pages from IndexNow


SeNioR-

Recommended Posts

I suggest excluding pages like /modcp/ from IndexNow as the crawler can't access them anyway.

207.46.13.11 - [07/Jan/2023 +0100] "GET /modcp/reports/4120/ HTTP/1.1" 403 11672 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
Link to comment
Share on other sites

ok thanks, Bing Search shows me that this page was sent by IndexNow. Don't know why, maybe a single case.

Could contain: Text

55 minutes ago, Daniel F said:

I guess you mean to add an no index directory to the robots.txt file?

 

no, not that 🙂

Edited by SeNioR-
Link to comment
Share on other sites

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...