Jump to content

After update last patch - Lighthouse was unable to download a robots.txt file


Recommended Posts

Posted

This test is from your site - but is also from many too sitesCould contain: Page, Text

Please check - this degrades the SEO performance of sites.

 

Posted
6 minutes ago, Matt said:

I'm not sure what we can do about that, it must be a page speed tool issue.

Probably so, some problem in PSI.
Now  I tested sites with other platforms - and popular sites. Everyone has this issue.
Apparently, it is a coincidence in the time of updating your patch, and the appearance of an issue in PSI.
You can close this topic.
Sorry for the trouble.

Posted

I found a solution in an article about this.
Add in robots txt

Disallow: /cdn-cgi/

and issue will not be there.
It worked OK for me.

Posted

But why do it though?  These are supposed to be recommendations to help you improve your user experience. I don’t see how doing that does anything like this. The tool clearly has a bug.  If it’s not actually helping improve the UX, the fix is a placebo. 

Posted
10 hours ago, Randy Calvert said:

But why do it though?  These are supposed to be recommendations to help you improve your user experience. I don’t see how doing that does anything like this. The tool clearly has a bug.  If it’s not actually helping improve the UX, the fix is a placebo. 

The error message that occurs when testing with Lighthouse, which causes a drop of 8 points in the SEO metric, "robots.txt is not valid - Lighthouse could not download robots.txt file", is a common problem when using a network for content delivery (CDN) such as Cloudflare.
This supplement in robots txt, I mentioned eliminates this issue. Probably Lighthouse changed (updated) their version and introduced a bug.
Another issue related to the SEO metric is that 2 zones are indicated in a guest cookie message, for poorly sized tap and click zones. And when viewing these areas, they are not links to touch, but part of the text - which is also a consequence of this bug.
Add-on in robots txt removes and this issue.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...