Jump to content

Large community? You have a problems with sitemap!


Numbered

Recommended Posts

@Matt Thanks for taking an interest in this. Will 4.3 support Google Tag Manager I.e. the latest Google recommended method of adding Google Universal Analytics via Google Tag Manager?

At the moment I use the current input box for Google Analytics code for the Google Tag Manager snippet code which goes in the head section, then I create a new global front system/core template called GoogleBodyTags for the Analytics code snippet which gets included as close as possible to the opening body tag of the globalTemplate, which seems a bit fiddly. 

 

 

Link to comment
Share on other sites

3 hours ago, Matt said:

I've had a quick peek at your site, and I think you're probably being penalised based on your keywords. Given the prevalence of 'fake news' and Google/Facebook being pressurised to do something about those sorts of sites, your keywords 'community forum for members to discuss the paranormal, conspiracies, ufos, games, reviews, technology, politics' probably are a bit of a red flag.

That's not quite fair, I know it's nothing to do with IPS, but who is Google/Facebook to say what is real news and what is not? Seems like it's forcing people to think one way or the other, very Orwellian. I know I had a ticket in about a year ago for some sitemap problems, and I wonder if it might had never been truly fixed? It always worried me because it wouldn't set to defaults. I keep an eye on Xenforo since I used them for years, and I haven't seen threads like this over there much. So it's starts to panic me like "should I switch back to Xenforo 2 for improvements"? But I know you guys are doing what you can.

Link to comment
Share on other sites

22 minutes ago, nodle said:

That's not quite fair, I know it's nothing to do with IPS, but who is Google/Facebook to say what is real news and what is not? Seems like it's forcing people to think one way or the other, very Orwellian. 

I feel a new conspiracy theory is just being developed. Perfect for your site! :lol: 

Link to comment
Share on other sites

  • Management
14 hours ago, nodle said:

That's not quite fair, I know it's nothing to do with IPS, but who is Google/Facebook to say what is real news and what is not? Seems like it's forcing people to think one way or the other, very Orwellian. I know I had a ticket in about a year ago for some sitemap problems, and I wonder if it might had never been truly fixed? It always worried me because it wouldn't set to defaults. I keep an eye on Xenforo since I used them for years, and I haven't seen threads like this over there much. So it's starts to panic me like "should I switch back to Xenforo 2 for improvements"? But I know you guys are doing what you can.

Google "Baader-Meinhof Phenomenon".

Quote

Baader-Meinhof is the phenomenon where one happens upon some obscure piece of information-- often an unfamiliar word or name-- and soon afterwards encounters the same subject again, often repeatedly.

Basically you are now tuned into believing IPS has an SEO problem, so you are seeing 'evidence' of that everywhere, whereas you are not really active over at xenForo so you do not notice the issues they may (or may not have).

Also: https://xenforo.com/community/threads/seo-poor-after-move-to-xenforo-from-ipb-help.129943/

Link to comment
Share on other sites

6 hours ago, Matt said:

Basically you are now tuned into believing IPS has an SEO problem, so you are seeing 'evidence' of that everywhere, whereas you are not really active over at xenForo so you do not notice the issues they may (or may not have).

Like I said I know it's not your IPS fault. Just like everyone else in this thread I came looking for an answer to "why". The link you posted here:

https://xenforo.com/community/threads/seo-poor-after-move-to-xenforo-from-ipb-help.129943/

Basically this was from him switching over and if you look at the date June 7th was way before all of this started happening with Google at the end of 2017. Mine site was fine until the end of December. Anyways it doesn't matter, I will just patiently wait until 4.3 comes out. I am interested with the changes to SEO that you made and will see if that makes a difference. :)

Link to comment
Share on other sites

Hey @nodle I know you mentioned the drop off on Google page displays and a concern over sitemap.  Do you see a trend (either increasing or decreasing) in your Google Analytics of the number of actual users and sessions over the same time frame? 

The sitemap may show more or less pages being indexed, but what ultimately matters is the number of visitors and real engagements.  

Link to comment
Share on other sites

8 minutes ago, Joel R said:

Hey @nodle I know you mentioned the drop off on Google page displays and a concern over sitemap.  Do you see a trend (either increasing or decreasing) in your Google Analytics of the number of actual users and sessions over the same time frame? 

The sitemap may show more or less pages being indexed, but what ultimately matters is the number of visitors and real engagements.  

No same visitors as always, we have our regulars. My site is not a large forum with tons and tons of people. It's been around for a long time though and always sat at number one spot for many years, then out of the blue at the end of last year, it starts to drop off Google all together. No decrease in regulars etc. Just strange, then I run across this thread and I thought maybe it was related like everyone else did.But I am sure it is like @Matt said, it's just the new way of Google handling things I guess.

Link to comment
Share on other sites

40 minutes ago, nodle said:

No same visitors as always, we have our regulars. My site is not a large forum with tons and tons of people. It's been around for a long time though and always sat at number one spot for many years, then out of the blue at the end of last year, it starts to drop off Google all together. No decrease in regulars etc. Just strange, then I run across this thread and I thought maybe it was related like everyone else did.But I am sure it is like @Matt said, it's just the new way of Google handling things I guess.

What does Google analytics show you? Any changes in number of users or sessions? 

Link to comment
Share on other sites

I think it’s just Google doing it’s ‘thing’.

I’m pretty sure this has little to do with sitemaps. I process all my sitemaps each night with a simple in-house script for my 500k or so topics. It only takes a few minutes to run. I have been doing it this way for years.

All the links show up in Google as being in the sitemap. But about a year ago, Google started dropping them from the index by the hundreds of thousands. Also my home page key words used to be in the top 3 at Google, now they've been lowered into obscurity. It’s not blacklisted. Some links are still there but just not ranking.

Yet, if I post something on YouTube or set up an empty store on a popular ecommerce site, all of a sudden those links are up at the top of some search results. So, it may just be Google on yet another new algorithm deciding who gets to see what.

Edited by prupdated
Link to comment
Share on other sites

On 1/30/2018 at 12:42 AM, Lindy said:

Again, Google cleaned house in 2017 - many sites got hit, including those using Wordpress, vB, etc. There's just too many variables to get all up in arms over one component: a sitemap. A sitemap, even the complete absence of a sitemap is not going to cause your indexed pages to drop. If this has happened to you, it's because Google penalized you and dropped your content. A sitemap is also not a magic wand and Google will not consider it the be-all for indexed content. A sitemap is a good gap filler and provides Google additional insight, but most content is still indexed organically through links.

4.3 will provide for the mentioned improvements to the sitemap. Given 4.3 is so close, I'm sorry, but it would not be worth re-engineering those improvements for 4.2 and again, it's not going to be the magic solution you believe it to be. Unfortunately, I can only suggest focusing on the above list as one or more of those items is the likely culprit for your site being dropped.

My sitemap has everything in it. It's updated daily. Still I got slammed pretty good.

Years ago, I probably had 90% of my sitemap indexed. Now it's less than 10%. Nothing much has changed with the actual type of content. I've had ssl for years. And redirects from that have long since cleared.

So Google basically says most of my site isn't worthy of it's index. In the old days, it would be the other way around - 900k submitted and 850k indexed.

5a832cd123b76_ScreenShot2018-02-13at11_53_37AM.png.d97cad3042dfe713484b9538a1831090.png

 

5a832cd2a3473_ScreenShot2018-02-13at11_48_57AM.thumb.png.7aeb4363f7273b8aa37c5cfc9d167a4d.png

 

I don't think a special formula of sitemaps and meta tags are the ultimate fix here. It appears to just be Google deciding what's important.

The burden might just fall on the webmaster doing something their content versus Google's algorithm.

Here's a pretty decent article going into the Fred update with some ideas on what to do about it:

3 Examples of Impact From the March 7, 2017 Google Algorithm Update (AKA Fred)

https://www.gsqi.com/marketing-blog/march-7-2017-google-algorithm-update-fred/

Also see the Goole Quality Rater Guidelines:

https://static.googleusercontent.com/media/www.google.com/en//insidesearch/howsearchworks/assets/searchqualityevaluatorguidelines.pdf

Edited by prupdated
Link to comment
Share on other sites

9 minutes ago, Morgin said:

Anyone check with Bing? Maybe it's gone up with Bing. Bing is usually pretty happy to index whatever!

Bing seems happy with the way things are:

 

5a8366bc21b58_ScreenShot2018-02-13at4_27_48PM.thumb.png.59d79721d24bcd1949212702f82f8ae7.png

5a8366ba84b89_ScreenShot2018-02-13at4_28_17PM.png.548ee4375a495ec5ae5a37be9afa6965.png

In Google, the site has practically been buried.

Edited by prupdated
Link to comment
Share on other sites

The new Search Console has some useful tips and offers a few clues.

https://support.google.com/webmasters/answer/7440203

On one of my sites, entirely powered by IPS (Pages, mostly Gallery, no Forums), it says 2.5k out of excluded 6.5k urls were excluded for being duplicate pages without canonical tags.

Looking at the list, a lot of them are as a result of Google following various sort and display options for Gallery albums in Profile view, but these are albums that don't exist because the user is brand new and hasn't even submitted anything yet. I'd exclude them too!

Another 1.8k Pages were marked as excluded for being Alternative page with proper canonical tags.

Again, looking at the list, the urls seem to relate to different sized versions of Gallery images.

380 were excluded for Excluded by ‘noindex’ tag.

These appear to be various Pages Articles urls appended with options such as "sortby=record_views&sortdirection=desc&page=1&d=1&advancedSearchForm=1".

Another 409 were marked as "Crawled - currently not indexed: The page was crawled by Google, but not indexed. It may or may not be indexed in the future; no need to resubmit this URL for crawling." Very informative! These seem to be urls of Gallery images and page articles.

Another 300 or so were excluded for being soft 404s, mostly calendar events.

One area I've had a lot of Google errors in the past is with Tags not being crawlable, on one site in particular the majority were Tags on Blogs, supposedly fixed in a future release according to a ticket at the time.

Link to comment
Share on other sites

I've got a ton of tags generated search pages with noindex so I think it may be a missed opportunity for some traffic. When you click on the the tag, it brings up a search page of that keyword. I would add a little SEO to that and at least change the generated page title to include Search results for "tag/word" or something like that and also meta description. There are a lot of internal links from those search pages generated from clicking on a tag that could create more indexing. 

You could do something similar with additional pages and just add page 1, page 2, etc., into the title and meta for these search pages rather than just excluding it all. https://www.mysite.com/search/?tags= details&sortby=newest&page=1 and 2 and so on as well.

Link to comment
Share on other sites

In my long SEO experience, it is ALWAYS better to have a 301 redirect for any links in the index, rather than leave them not working. In fact, this is exactly why google is showing these links to you, so you can fix this.

In that spirit, in my case many of my links with issues have to do with the /members directory, adn look like this:

http://www.celiac.com/gluten-free/members/page__sort_key__joined__sort_order__asc__max_results__20__quickjump__E__name_box__begins__name__E__st__320

Because I don't see that the site uses /members anymore, I created a 301 redirect like this:

Quote

RedirectMatch 301 ^/gluten-free/members(.*)$ http://www.celiac.com/gluten-free

My question is: Am I correct that /members is no longer in use by the board? Obviously I don't want to create an issue with this redirect.

Edited by sadams101
Link to comment
Share on other sites

3 minutes ago, SeNioR- said:

@AlexWebsites I've done it before - for tests, I've improved the seo for tags and indexed well. I have a modified search.php file somewhere on the disk

I would be willing to test if you like on one of my sites. All these little SEO tweaks in this topic (like this with tags) could really improve organic traffic. 

Link to comment
Share on other sites

2 hours ago, bfarber said:

Google disagrees about allowing search result pages to be indexed

https://www.mattcutts.com/blog/search-results-in-search-results/

 

Thanks for sharing that article but its over a decade old... I just think because search results in IPS are so robust that tags can drive new pages with very good dynamic content, could be something to think about.

Here's an article along the same lines and it does seem to be frowned upon but again, maybe additional pages created by tags without a search box would be something. http://www.thesempost.com/googles-algo-removes-internal-search-results-pages/

Edited by AlexWebsites
Link to comment
Share on other sites

So is it ok to redirect /members to the board's index page? Is /members no longer used by the board, including the search engine, links, etc? Just want to be sure my redirect doesn't break anything, but to me it looks like /members is no longer used.

Also, and I hope someone didn't already bring this up here, but I am seeing these type of links in google's index and they are causing errors:

http://www.celiac.com/gluten-free/topic/84513-descriptions-of-dh-types/?do=embed

Should these be in a robots.txt or have a rel="nofollow" ?

Link to comment
Share on other sites

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...