SEO Rankings flying all over the place, and why is this..

[realmaverickuk]

deleted due to repeating myself.

[realmaverickuk]

And Steven, the issue has gone away. I just checked. However your search box is gone?

edit: by issue, I mean the anchors, are now simply #entry123, which isn't a string, it's just an anchor.

[Matt]

Thanks, that's very useful. That kind of thing I can easily do. I'll add an option to use no follow for tags and clean up pagination links.

[realmaverickuk]

Great, thanks Matt.

Touch off topic, but similar to the tags fix you made, ensuring all canonical pages are clearly defined. This would be a useful addition to the Marketplace too. Anywhere that features sorting options, or any kind of alternate page, would benefit from the canonical tag. Ultimately it just helps the engines understand which page they should rank, and helps eliminate duplicate content issues.

I had Brandon D make the fix on my website recently, but I'm sure many others wouldn't have means to implement it.

Anyway, I'm pretty sure IPB will soon be fully optmised. The only difficult optimisation, that I feel would be of benefit, is the code to content ratio. Though I bet you could easily delete the big white space that's in the source code?

[Alfa1]

If I may address some points:

- stop using URL that are not human readable (ipb's page/lastmessage urls look awful )

You really don't need to route all URLs via human readable url. Especially content you do not want spidering heavily. Unless you have lots of /findpost/p/123/st/123 which is not really human readable.

But if it gets spidered, then you are in trouble. Highly complex urls with loads of variables should be avoided.

- if possible, you should even try to avoid thread ID is URLs (like wordpress does)

Sadly not possible. At least without having topic/test-101/ (assuming there are 100 topics called test already). Also, I think that in general comparing Wordpress is like comparing apples to oranges. Even the most busy Wordpress blog may only create 30-40 new articles a day. A busy forum can create thousands. And don't discount that with a blog *you* are almost the sole content creator so you can tailor your content better.

I disagree. Why would it not be possible? Why not just add thread IDs to duplicate thread titles?
Which leads to another issue: duplicate thread titles are not good. Why not report it to the moderator in charge, so it can be changed?

- stop havin everything under /forum/ or /topic/ , at least have an option not to.

Honestly, I'm not convinced this is a problem. There's no way of knowing for sure without a lot of recoding and seeing what happens.

Look at what google does itself. It uses very logical directories that define what something is.
docs.google.com
video.google.com
news.google.com/news/
etc.

- stop pagination links on top of the page

But they're convenient for humans.

And therefore they are good for spiders.

I'm just spinning my wheels here, but if we assume tags are indexed better because they have more back links because for each tag there is (tag x ( topics x pages) ), would it help to have a "Back to first page" link on all subsequent pages somewhere? This might increase the back links to the root page where the most keywords are.

tags get indexed well, because of the link structure in combination to the keyword relevance. Basically tags provide a page filled with keyword relevant links to pages filled with the same keyword and related content. This method has worked well since 1997.

/topic/123-hello-world/?sort=posts

/topic/123-hello-world/page-2/?sort=desc

Dont make any kind of sorting urls accessible to guests at all, except if you can sort by content like tags do. Content is king.

That reference you mention brings me to:

community.invisionpower.com/topic/360258-ipboard-the-forum-will-be-dead-within-5-years/ as the top result which is fine.

The problem with the tags, is that Google are using them as the core reference to the thread, instead of using the thread itself as the first page entry. this will lose you traffic.

Although, be clever how you use them, tags.

If your thread is called:

"dancing cats"

Then don't use the tags exactly as:

"dancing cats"

Do a tag like:

"cats that might be dancing"

My testing has found that doing it that way does not detract Google losing the thread for the original entry "dancing cats, but you will still attract traffic for people typing in "cats that might be dancing", and why not, it is still traffic.

Hope that makes sense.

Tags should be as content relevant as possible. If the thread is about dancing cats, then it should be tagged 'dancing cats'.
The problem lies in the weakness of threads.

[list=1] [*]Tags: These are generally not useful and usually do more harm than good. I don't believe it's an issue for IPB to fix, as there is nothing really to be fixed as such. The major SEO plugins for Wordpress, add noindex to tag pages.

[/list]

Tags drive a massive amount of traffic to vbulletin sites. Millions a year on my site. Its a very strong feature if implemented well.

The most likely cause of tags outranking main content, is because of the fact the tags have more links going to them than the thread itself.

There is no reason why threads should not get strong SEO. But this is something that can be improved.Lowering tag SEO seems counterproductive, as you want your search results in google ranking higer than others.

[realmaverickuk]

Tags drive a massive amount of traffic to vbulletin sites. Millions a year on my site. Its a very strong feature if implemented well.

Tags can drive a lot of traffic, if the pages themselves are well optimised. IPB tag pages are not well optmised at all. But they can also lead to a lot of thin, duplicate content, which is a big issue these days. Websites with more authority can get away with a lot more, so tags must be used with caution, especially on less authoritive websites. Don't make the assumption, that all websites are treated equally, because they are not. IPB needs to be a safe tool to use, out of the box. With advanced features that can be activated by advanced users.

The problem lies in the weakness of threads.

Partly. But it's also clear, that if you're linking to the tags page with "dancing cats", over many pages and threads of a forum, that it's got a good chance of out ranking the thread for Dancing Cats. It's logical, the more links you point to something, the more authority for that term, it will gain.

[Rimi]

Pages ARE being demoted - fact.

It's hardly a fact if it's only happening on your site. :D I'm still looking for a site that does what yours is doing.

You want no index on page 2+? Hmm something like

<if test="$this->request['st'] > 10">
<meta name="robots" content="noindex" />
</if>

I made that up. Don't use it.

[realmaverickuk]

FWIW, VB tags aren't well optmised either. Nor do they rank well for ANY of the keywords and websites I've just tested.

Take VBSEO.com, tag: optimisation

https://www.google.co.uk/search?sugexp=chrome,mod=0&ix=h9&sourceid=chrome&ie=UTF-8&q=site%3Ahttp%3A%2F%2Fwww.vbseo.com%2F+htaccess#hl=en&sclient=psy-ab&q=site:http%3A%2F%2Fwww.vbseo.com%2F+optimisation&oq=site:http%3A%2F%2Fwww.vbseo.com%2F+optimisation&aq=f&aqi=&aql=1&gs_l=serp.3...6270.8373.1.8484.12.12.0.0.0.0.109.720.11j1.12.0.eish.1.0.0.WT6SysE2GUw&pbx=1&bav=on.2,or.r_gc.r_pw.r_cp.r_qf.,cf.osb&fp=d6c31d58b45f9e3b&ix=h9&biw=1532&bih=1232

http://www.vbseo.com/tags/optimization/

This page doesn't appear in the first few pages of the results that I checked.

[realmaverickuk]

It's hardly a fact if it's only happening on your site. :D I'm still looking for a site that does what yours is doing.

You want no index on page 2+? Hmm something like

<if test="$this->request['st'] > 10">

<meta name="robots" content="noindex" />

</if>

I made that up. Don't use it.

I made that up, don't use it.... haha, made me smile.

If Steven wants to noindex page 2+, which I don't think he does, I strongly, firmly, suggest that he doesn't do it. This is not the solution.

[Rimi]

Just checked the code and it works. Silly thing to do though. Also it's bad code since I wrote it.

[realmaverickuk]

Steven, I just noticed you've made a change. Not allowing guests or Google to use search or tags?

So as a guest or Google, I visit a tag page i.e http://www.makemoney... money at home/

I get an error, I checked the header, it's a 403 forbidden error. So you're sending Google to tons of pages, that it's not allowed to access.

This will
1. Confuse Google
2. Waste link juice
3. Waste your crawl allowance
4. Irritate visitors who aren't logged in

The issue when people mess with SEO, without understanding the implications, they do more harm than good.

Perhaps you do understand the implications but didn't think it through properly, I don't know. But you should seriously undo it, asap.

[Alfa1]

Tags can drive a lot of traffic, if the pages themselves are well optimised. IPB tag pages are not well optmised at all.

Yes, you are right.

But they can also lead to a lot of thin, duplicate content

Please explain what you mean by this.

Websites with more authority can get away with a lot more, so tags must be used with caution, especially on less authoritive websites. Don't make the assumption, that all websites are treated equally, because they are not.

Its logical that authoritative sites can get away with a lot more, because they are already scoring good on various points. But it still weighs in on the same calculation. Bad optimization hurts any site. So I dont agree with you here.

IPB needs to be a safe tool to use, out of the box. With advanced features that can be activated by advanced users.

Very true.

Partly. But it's also clear, that if you're linking to the tags page with "dancing cats", over many pages and threads of a forum, that it's got a good chance of out ranking the thread for Dancing Cats. It's logical, the more links you point to something, the more authority for that term, it will gain.

I think the error in thinking here is that its an issue that one page outranks the other page. If I google dancing cats, then both the thread as the tag page should come up. And only if the tag page has multiple topics listed.
I dont think its an issue that tag pages outrank thread pages. I honestly cant imagine how it can not outrank threads.
I think the main issue is that both need improvement.

Does IP.SEO sitemap submitter allow you to define weight to urls? If not, then this function should be added.

FWIW, VB tags aren't well optmised either. Nor do they rank well for ANY of the keywords and websites I've just tested.

Take VBSEO.com, tag: optimisation

https://www.google.co.uk/search?sugexp=chrome,mod=0&ix=h9&sourceid=chrome&ie=UTF-8&q=site%3Ahttp%3A%2F%2Fwww.vbseo.com%2F+htaccess#hl=en&sclient=psy-ab&q=site:http%3A%2F%2Fwww.vbseo.com%2F+optimisation&oq=site:http%3A%2F%2Fwww.vbseo.com%2F+optimisation&aq=f&aqi=&aql=1&gs_l=serp.3...6270.8373.1.8484.12.12.0.0.0.0.109.720.11j1.12.0.eish.1.0.0.WT6SysE2GUw&pbx=1&bav=on.2,or.r_gc.r_pw.r_cp.r_qf.,cf.osb&fp=d6c31d58b45f9e3b&ix=h9&biw=1532&bih=1232

http://www.vbseo.com/tags/optimization/

This page doesn't appear in the first few pages of the results that I checked.

Come on. Seriously. If you really think that this proves anything, then think again.
You could just as well have used the word 'google' or 'sex'. These would also not come up. Not even if you use wikimedia or wordpress, which have excellent SEO.
Prove your point with somethign thats not supergeneric. Plus vbSEO is an addon product, and vbseo.com is not repesentable for normal vbulletin sites.
Using google.com will give you results that apply to more people.

Sure vbulletin tag pages can be optimized. But they are 100 times better in SEO than IPB.
Reasons:

• linking structure: how do all pages link to each other throughout the site and how are tag pages positioned?
• thread preview on hover: hiding thread preview from bots is a major issue with IPB. It should at least be an option to display topic preview on hover and do away with the current preview function.
• url syntax
• the way tags are displayed on topic view/tag pages
• forum view / tag pages do not allow large amounts of tags to be listed, because tags are displayed under the thread title. Try that with 80 tags. Its not possible.
• alt tags are often not used or include text that is not content relevant. One small example: Hovering over a tag will gives you an alt text of ‘Find more content tagged [tag name]’. So for every tag your site has, the keywords ‘find more content’ are added. It would be best for SEO to remove this and only have the tag name in the alt text. I have 27000 tags. Lets say each is used 100 times. That makes 2.7 million mentions of irrelevant keywords.
  

[Enkidu]

OK. The isssue with your website is clear:


let's do:

spinacoin review site:makemoneyforum.com

clearly you have converted your website from another software package, likely WP. Google is trying to re-index your website. Some of your original wp links are returning 200 so google still think they are valid. However, the others that don't are returning 401 instead of 404. Google will take sometime to discard those from its database but eventually it will happens.

Now look at this:



your sitemap is indexed by google? how did that happen? did you link to it from somewhere? notice that this is a WP sitemap. Do you have two software packages running on the same place?


looking at your robots.txt:

User-agent: *

Allow: /

Disallow: /Suggests/

Disallow: /user/

This is robotos.txt? why do you block user?

let's make the search term broader:

site:makemoneyforum.com

your index page has rel author somewhere pointing to your google profile. why is that?

I can continue doing that but you got the idea. It is not IPB fault but rather yours.

[bfarber]

I try not to weigh in on SEO topics (doesn't mean I'm not listening though ;) ), but a few things caught my eye that I wanted to comment on.

Great, thanks Matt.

Touch off topic, but similar to the tags fix you made, ensuring all canonical pages are clearly defined. This would be a useful addition to the Marketplace too. Anywhere that features sorting options, or any kind of alternate page, would benefit from the canonical tag. Ultimately it just helps the engines understand which page they should rank, and helps eliminate duplicate content issues.

I had Brandon D make the fix on my website recently, but I'm sure many others wouldn't have means to implement it.

Anyway, I'm pretty sure IPB will soon be fully optmised. The only difficult optimisation, that I feel would be of benefit, is the code to content ratio. Though I bet you could easily delete the big white space that's in the source code?

Firstly, each application has to do these things independently. If you file a bug in the IP.Downloads tracker category, I'd be more than happy to look at the canonical tag usage in the download manager for the next release (which is what powers our marketplace). :)

Secondly, I don't think white space of any form will have much effect at all on content to code ratio.

I disagree. Why would it not be possible? Why not just add thread IDs to duplicate thread titles?

Which leads to another issue: duplicate thread titles are not good. Why not report it to the moderator in charge, so it can be changed?

You must remember that with a product like Wordpress, the admin (who cares about SEO) is creating the content pages and can of course think about the titles, clearly define them so that they are relevant and unique, and so forth. With a forum product, users (who do not care about SEO of your site one bit) are submitting topics, and often use duplicate titles. It is not feasible to omit a thread ID by default in these URLs.

Look at what google does itself. It uses very logical directories that define what something is.

docs.google.com

video.google.com

news.google.com/news/

etc.

I wouldn't argue about the usability or "prettiness" of such URL structures, however from a technical POV this is very challenging because it requires the users to set up server configurations (and it's not really possible from a PHP script to check they are correct and handle them when they are incorrect). It is easy to say "well that would be the admins fault", however we still have to provide technical support for our product. ;)

That is not to say we'll never see a possibility like this, but it is extremely challenging to implement something like this in a manner that won't break sites when they're not set up 100% properly.

Dont make any kind of sorting urls accessible to guests at all, except if you can sort by content like tags do. Content is king.

I disagree that we should lessen the experience for guests who are human to improve the experience for guests who are not. Reducing the functionality of the software to get rid of a couple URL variations does not sound like the best answer to me.

[Alfa1]

You must remember that with a product like Wordpress, the admin (who cares about SEO) is creating the content pages and can of course think about the titles, clearly define them so that they are relevant and unique, and so forth. With a forum product, users (who do not care about SEO of your site one bit) are submitting topics, and often use duplicate titles. It is not feasible to omit a thread ID by default in these URLs.

Point taken. The exact same issue is the case with wiki software. There are no id numbers in wiki titles. So that creates the same problem for user generated content. For example Vaultwiki reports duplicate threads, so staff can act on it.

I wouldn't argue about the usability or "prettiness" of such URL structures, however from a technical POV this is very challenging because it requires the users to set up server configurations (and it's not really possible from a PHP script to check they are correct and handle them when they are incorrect). It is easy to say "well that would be the admins fault", however we still have to provide technical support for our product. ;)

It may be challenging, but surely possible. vbSEO and NuSEO are addon products that provide such functionality to vbulletin and in the case of NuSEO to a series of other products like photopost.
NuSEO was coded in a few months. I know because I worked closely with the developer.

Surely if they can do it, IPS can do it as well.

That is not to say we'll never see a possibility like this, but it is extremely challenging to implement something like this in a manner that won't break sites when they're not set up 100% properly.

Yes, I understand this is a major concern and a headache that no one needs. But this can be safeguarded by making such functions optional, so the URL rewrite rules have to be turned on, configured and a .htaccess file needs to be added, so IPS is able to control the urls.

Honestly, SEO is a major topic for IPS. URL syntax is a problem. It seems to me that adding Custom rewriting is hard to avoid.

I disagree that we should lessen the experience for guests who are human to improve the experience for guests who are not. Reducing the functionality of the software to get rid of a couple URL variations does not sound like the best answer to me.

Yes, I disagree as well that you should lessen the experience for guests. But thats not what I am saying. My suggestion is: Keep the functionality. Loose the urls.
Load the content in the same page and reorder with javascript.

[realmaverickuk]

[color=#282828][font=helvetica, arial, sans-serif]Come on. Seriously. If you really think that this proves anything, then think again.[/font][/color]

[color=#282828][font=helvetica, arial, sans-serif]You could just as well have used the word 'google' or 'sex'. These would also not come up. Not even if you use wikimedia or wordpress, which have excellent SEO[/font][/color]

What you said, just made no sense at all. Does vbseo have a tag of sex? I was doing a site specific search, for a tag that vbseo actually have. My point was, if the tag page was so well optimised, it would have appeared on the first few pages, when searching specifically for that keyword, on a site specific search! But the tag pages aren't well optimised and so it didn't.

VB tags are better optimised than IPB, in some of the ways you've explained, but IMO they leave a lot to be desired. I'd still personally rather deindex the lot of them anyway.

In IPB, the titles, H1 etc of tag pages are terrible. So it's surprising that Stevens tag pages are competing with his threads. The only real logical explanation, is that it's due to the fact, the thread has many links, pointing to the tag page, with optimised anchor text for that keyword.

[Steven UK]

It's hardly a fact if it's only happening on your site. :D I'm still looking for a site that does what yours is doing.

Give me a list of sites, and I will find you some. The fact that Google are not organising threads is global, not just us.

[realmaverickuk]

The issue with URLs, isn't really the ID. Although it would be better for the ID to appear at the end of the posts title, rather than at the beginning.

Currently, topics all come under /topic/ which doesn't portray the relationship of the pages and forums well. Content in folders, helps Google understand the relationships between pages, just as the breadcrumbs do. Naturally you'd have /feedback/ipboard/post-title/page-2/

OK. The isssue with your website is clear:

let's do:

spinacoin review site:makemoneyforum.com

clearly you have converted your website from another software package, likely WP. Google is trying to re-index your website. Some of your original wp links are returning 200 so google still think they are valid. However, the others that don't are returning 401 instead of 404. Google will take sometime to discard those from its database but eventually it will happens.

Now look at this:

[img]

[/img]

your sitemap is indexed by google? how did that happen? did you link to it from somewhere? notice that this is a WP sitemap. Do you have two software packages running on the same place?

looking at your robots.txt:

User-agent: *

Allow: /

Disallow: /Suggests/

Disallow: /user/

This is robotos.txt? why do you block user?

let's make the search term broader:

site:makemoneyforum.com

your index page has rel author somewhere pointing to your google profile. why is that?

I can continue doing that but you got the idea. It is not IPB fault but rather yours.

If you look, he's using Wordpress for his blog and IPB for the forum, so naturally the Wordpress posts will show a 200 header code :S

[realmaverickuk]

https://www.google.co.uk/search?sugexp=chrome,mod=5&sourceid=chrome&ie=UTF-8&q=maybe+you+could+give+us+a+change+log%3F

This demonstrates the big issue quite well.

Take a look at the first two results. They are both for the same thread, page 1 and page 2. But both are being seen as completely independent. That's why it states a different number of posts and authors for each. Google thinks each is a different topic. That much is certain.

Now, the fact Google thinks each is a separate topic, is a big issue, the big issue being both have the same title. Page 1 and page 2 are now competing for the rank for the keyword. But more importantly, duplicate titles, tells Google that you've created 2 pieces of content, on the same subject. Something that Google Panda was designed to penalise against.

This could be doing a lot more harm than just Google struggling to know which page to rank.

[Steven UK]

OK. The isssue with your website is clear:

clearly you have converted your website from another software package, likely WP. Google is trying to re-index your website. Some of your original wp links are returning 200 so google still think they are valid. However, the others that don't are returning 401 instead of 404. Google will take sometime to discard those from its database but eventually it will happens.

It is far from 'clear', and neither is it 'clear' that we have converted the website from another package, which is 100% not accurate. This forum has always been IPB, we simply added a WP to the front end, and placed the forum into /forums/ which countless other website owners do.

your index page has rel author somewhere pointing to your google profile. why is that?

The author issue is nothing to do with anything we are talking about here.

I can continue doing that but you got the idea. It is not IPB fault but rather yours.

Really? So the fact that the following is getting indexed instead of threads:

http://www.makemoneyforum.com/index.php?app=core&module=search&do=search&andor_type=&sid=288ad9b1d1a8606dd6f809070c872fc1&search_tags=www.spinacoin.com&search_app_filters%5Bblog%5D%5BsearchInKey%5D=entries&search_app_filters%5Bblog%5D%5Bentries%5D%5BsortKey%5D=date&search_term=&search_app=ccs

Is down to me?

Or the fact that Google have no reference point when 'stacking related threads' is down to me?

The fact that Matt himself has admitted that he does not know why Google does not reference posts like other software, is down to me?

No thank you.

[Steven UK]

This could be doing a lot more harm than just Google struggling to know which page to rank.

It is doing more harm, especially when you throw some other variable pages into the mix, like users, searches, and much more which is getting indexed instead of the intended threads. And even if those other variables were not in the equation, you could easily have a duplicate content issue. X that by 20,000 IPB users, and Google could easily just blacklist the core coding of the software, with a flick of an algorithm change, based on a virtual spider report.

[realmaverickuk]

Here's another example: https://www.google.co.uk/search?sugexp=chrome,mod=5&sourceid=chrome&ie=UTF-8&q=Post+Edit+History+-+REALLY+NEED+THIS

Imagine threads with 10+ pages. Pretty major problem here. I wasn't even aware of it until today. I guess because we don't really utilise our forum, it's more of an afterthought. But this is pretty major IMO.

[Steven UK]

This is exactly what I have been saying, Maverick.

Apparently though, nobody else is seeing these results, so you can blame me for all the duplicate content, and when your site gets de-indexed for spamming the search engines ;-)

Seriously though, was this noticed before the latest updates, was this issue there then? because I have found this to be sporadic, probably due to all the variables that are also getting indexed (searches, etc), and depending on how the SERPS decides to filter, which is not static, when offering them so much to index.

[realmaverickuk]

This is exactly what I have been saying, Maverick.

Apparently though, nobody else is seeing these results, so you can blame me for all the duplicate content, and when your site gets de-indexed for spamming the search engines ;-)

Seriously though, was this noticed before the latest updates, was this issue there then? because I have found this to be sporadic, probably due to all the variables that are also getting indexed (searches, etc), and depending on how the SERPS decides to filter, which is not static, when offering them so much to index.

I think everybody become fixated on the specific issue you were having with the page__, coming from the failed upgrade.

But the issue with the lack of relationship of the pages, is a very real one and present in all installations.

Something interesting I've just noticed, Google: https://www.google.co.uk/search?sugexp=chrome,mod=5&sourceid=chrome&ie=UTF-8&q=SEO+Rankings+flying+all+over+the+place%2C+and+why+is+this..

Searching for the exact match of this topic title, returns no results for IPB. Perhaps because there are 9 pages, all with the same topic, the forum has now been penalised for the term? That's just a guess, as I've not done much investigating but will look in to it some more.

[Steven UK]

I think everybody become fixated on the specific issue you were having with the page__, coming from the failed upgrade.

But the issue with the lack of relationship of the pages, is a very real one and present in all installations.

Something interesting I've just noticed, Google:

https://www.google.c...d why is this..

Searching for the exact match of this topic title, returns no results for IPB. Perhaps because there are 9 pages, all with the same topic, the forum has now been penalised for the term? That's just a guess, as I've not done much investigating but will look in to it some more.

That's right. This whole thread is no longer even indexed.