Invision Community 4: SEO, prepare for v5 and dormant account notifications By Matt Monday at 02:04 PM
Gauravk Posted April 26, 2016 Posted April 26, 2016 Wondering why IPS4 doesnt come with robot.txt? In my site I am still having old one and I think it need an update to manage certain stuff that we really dont want to get crawl like empty profile pages, admin area and few testing pages. Not sure wht all we have added overtime in 3.X.X so appreciate if someone can help cleaning in below robot.txt please with bare minimum entries, as we are seeing lot of crawl error from GWT, and doubting it may be because of this wrong robot.txt. Thanks. User-agent: * Disallow: /admin/ Disallow: /cache/ Disallow: /converge_local/ Disallow: /hooks/ Disallow: /ips_kernel/ Disallow: /retail/ Disallow: /public/style_captcha/ Disallow: /discover/ Disallow: /profile/ Disallow: /latest Disallow: /updates Disallow: /classy Disallow: /index.php?app=core&module=task Disallow: /index.php?app=forums&module=moderate§ion=moderate Disallow: /index.php?app=forums&module=extras§ion=forward Disallow: /index.php?app=members&module=chat Disallow: /index.php?&unlockUserAgent=1 Disallow: /*&module=search Allow: /*&search_tags= Disallow: /*app=core&module=global§ion=reputation Disallow: /*app=core&module=usercp Disallow: /*app=members&module=messaging Disallow: /*&p= Disallow: /*&pid= Disallow: /*&hl= Disallow: /*&start= Disallow: /*view__getnewpost$ Disallow: /*view__getlastpost$ Disallow: /*view__old$ Disallow: /*view__new$ Disallow: /*view__getfirst$ Disallow: /*view__getprevious$ Disallow: /*view__getnext$ Disallow: /*view__getlast$ Disallow: /*&view=getnewpost$ Disallow: /*&view=getlastpost$ Disallow: /*&view=old$ Disallow: /*&view=new$ Disallow: /*&view=getfirst$ Disallow: /*&view=getprevious$ Disallow: /*&view=getnext$ Disallow: /*&view=getlast$ Disallow: /*?s= Disallow: /*&s=
tnn Posted April 26, 2016 Posted April 26, 2016 Here is some info laltroweb.it posted that may be of interest to you.My robots.txt fpr ipb4
Recommended Posts
Archived
This topic is now archived and is closed to further replies.