410 instead 404 for deleted database items

[sobrenome]

I am trying to follow google guidelines to make my site better.

One of the gold rules is to have great content. And I suppose that an old topic not locked and without any reply is no use for no one. My site has more than 10 years old.

So I mass deleted about 5% of my topics, old and unanswered (about 5 thousand topics).

But google reduced the organic traffic about 17% after the cleaning.

In webmaster tools, google is warning about mass 404 erros, the cause for the traffic punishment.

So I want to suggest for IPS 4 to address to the search engine the code 410 instead of the code 404 for the deleted topics, calendar events, database items, etc.

It seems that google is now considering 410 relevant. On the past, 404 was fine, but now the crawler demands for more specific response.

[sobrenome]

I had 7.000 pending users and decided to turn on the auto pruner for pending users.

Google is also reporting mass 404 erros for users.

The next IPS 4 should also send 410 response for deleted users.

[Mark]

Google treats 404 and 410 the same (source). It's not warning you that you're doing anything wrong per se, it's just warning you that you've deleted content, so it's going to remove some of your URLs from it's index, which of course, may have a negative affect on incoming traffic if you were getting click-throughs to topics that have now been deleted. Naturally, this will happen whether it's 404 or a 410 header it gets.

More information: http://googlewebmastercentral.blogspot.co.uk/2011/05/do-404s-hurt-my-site.html

[sobrenome]

This article was written almost 3 years ago. Many things have changed in the algorithm.

The topics that I have deleted (old and unanswered) did not bring 20% of organic traffic, they did not get any organic at all.

I just think that the subject deserves better analysis.

Thanks

[Mark]

As I say, Google's documentation still says 404 and 410 are treated the same.

If you believe there is an issue, I'd be happy to take a look for you if you submit a support request providing access to your Analytics and Webmaster Tools accounts (mention this post), but a temporary decrease in traffic following deleting a significant amount of content is quite normal.

As general advice, I would not recommend deleting content ever, except of course it is offensive and you don't want it on your site. It is usually the topics that have been around for a long time that get the most traffic, so I disagree that old, locked topics are not good - those are often your best topics for search results. Don't delete them. :)

[sobrenome]

locked topics were not deleted, only open and unanswered.

The article in google was closed, and in the last comments (most recent) you can see some people complaining about traffic drop after deletion.

Another thing that must be considered is a Mass Deletion versus a Single Topic deletion per day or month.

I will open a ticket. Thanks!

[bfarber]

Yes, I'm quite certain that people complain about a decrease in traffic after deleting a large amount of content. That is normal, as has already been stated.

And yes there's a difference between mass deletion and deleting a one off topic here and there. The former, which is what you said you did, will lead to reduced traffic as Google tries to figure out why a bunch of links previously indexed no longer exist (404 or 410 header response) while one topic being deleted here or there is not terribly abnormal.

I wouldn't jump to the conclusion that deleting a ton of content (even if it appears to be empty content) on your site ultimately leading Google to point it out is a software issue, and that changing a 404 (not found) to 410 (gone) header is going to do anything about that. :)

[sobrenome]

Is there any report on how long does google take to understand that the mass deletion was intended instead of a software issue?

Are you pretty sure that a 410 response would not avoid this crawler mistake?

I am feeling betrayed by google for fowling the guidelines for better user experience and to have a dramatic ranking drop after doing it.

[bfarber]

I don't think it's actually a case of "intended vs software issue". It's a case of "10,000 pages on this site are now gone...let's reindex the site to see what's still valid and just push the down in search listings a little bit until that's done". If you step back for a moment and think about this from an end user perspective, or Google's perspective, that absolutely makes sense to do. I know you see it as a penalty because it affects YOUR site, but the reality of the matter is Google is trying to ensure the user experience for its visitors is as good as it can be.

[sobrenome]

The traffic is raising again, almost normal.

[bassangler]

Note that your traffic went down during the holidays, and is now back up again. Sometime correlation doesn't mean causation.

[opentype]

This article was written almost 3 years ago.

Here is a new statement from Google:

[sobrenome]

It would be nice with the IPS 4 could deal with deleted topics following the correct HTPP response.

[opentype]

As the video explains: there is no “correct” and “incorrect” way to serve these two error codes. 410 and 404 will basically have the same effect.

[Aiwa]

It doesn't seem like there is any benefit in the crawl engine that would justify differentiating 410 and 404.

If you deliver a 404, the content is preserved for 24 hours... That's actually nice when you consider that deleted content that may still be in the ModCP and not fully 'deleted' yet could potentially be restored. At which point a 404 would be the better response to give.