Google Crawl Errors

[Alex.amazingboy]

Getting heaps errors from Google Crawler, how to prevent them ?

For example :

[jair101]

It seems you have disabled guests to be able to see user profiles, so when google tries to crawl them it gives an error.

I added this to robots.txt:

Disallow: /user/

[Alex.amazingboy]

It seems you have disabled guests to be able to see user profiles, so when google tries to crawl them it gives an error.

I added this to robots.txt:

Disallow: /user/

Thanks mate and what about this :

More than 500 errors caused by this thing :D

[jair101]

Same, google is trying to use the search, which you have blocked. I have this line in my robots.txt, should help:

Disallow: /*&module=search

[Alex.amazingboy]

thank you :)