Search Engine Indexing Forum Pages

[brandoninsc]

I am just now getting some traffic to my site, but it is from pages that I have created and submitted the URL to Google Search Console.  I've gone from practically no search engine presence to approximately 3,000 impressions a day in about a month. Again, all from manual submissions to the search console, but perhaps the sitemap has helped as well.  However, most of the forum pages that I check through Google Search Console, they all show up as Discovered - Currently Not Indexed or just not found.  I can submit them, but they are never indexed.  I thought it was because the content may still be thin, but I can literally create a new page with thin content, submit that URL and it will be crawled within a day. Of the 170 or so pages that Google has indexed, I have manually created the vast majority of those. 

I've read some other comments about how Google doesn't have unlimited resources so it doesn't index everything on a sitemap, or what is submitted and I get that. I think i've heard it called a site's crawl budget, but it only seems to impact the forum (calendar and gallery) pages created by Invision, not the pages I create manually.  Just to be clear, my manual pages are pages that I write in PHP and add to the pages component of Invision. I submit those, and they are quickly indexed, whereas my forum topics have been submitted for days, weeks, months and are not indexed. 

Is this due to Google deciding that they don't want to index my forum pages, but quickly indexing any manual page I submit, or is there something in the Invision software that prevents indexing until a certain point or length? Is there a setting or requirement that I'm missing to get these moved out of the Discovered list and onto indexed? I'd like to start driving traffic to the forum topics, but if my topics are never indexed - that seems like a big problem.  What am I missing? 

 

[opentype]

This is mixing two topics: crawling and indexing. It’s better to address them individually or it gets messy and confusing.

“crawled, currently not index” is entirely Google’s decision. It does not depend on any settings and there is nothing you can do about these individual URLs. It’s Google’s judgement of the actual content and its relevance for Google’s users. 

Crawling everything in the sitemap should be fully automatic. You don’t need to submit anything manually. Technically, you don’t even need a sitemap. It just helps the process and avoids unnecessary crawler traffic. You didn’t say whether your forum topics are actually crawled successfully. If not, there could be technical problems. If they are crawled but not indexed, then everything you need to know will be in this list in the Search Console:

We could only speculate. The useful facts will be in that list. 

[brandoninsc]

Perhaps I'm not clear, or have a misunderstanding... but I am speaking specifically about the classification Discovered - Currently Not Indexed.  Most of my forum pages can be found there. 

I understand if Google crawls it, and doesn't want to index it for any reason.  What I cant understand is discovered and not indexed.   Whether is is through the sitemap, or manually submitted - they are never crawled.   However, I can make a non-forum page, on pretty much the same topic with a lot of the same words - and submit it, and its crawled and indexed within 24 hours. 

It seems very much to me that either Google has some rationale for not indexing forum pages, or there is something else preventing these pages from being indexed, and perhaps in my case - even crawling them at all.   If its thin content, I get it - but I have much thinner content submitted, crawled then indexed, quickly. 

[Nathan Explosion]

Quote

The Discovered - currently not indexed status means that Google knows about these URLs, but they haven't crawled (and therefore indexed) them yet.

Google probably gone for a cup of tea.

[Marc]

While they will usually be indexed relatively quickly, they will not be indexed instantly. If google knows they are present (as shown in your screenshot) they will index them eventually. We have no control over when google decides to do that of course

[brandoninsc]

I get that and understand that Google makes their decisions based on some nebulous criteria... my point here is, for some reason, my forum pages seem to be a significant variable compared to any other page I create, which are almost always indexed within a day. So while the responses here seem to indicate that there is no software setting or problem causing this, I was hoping that someone else has seen this and could explain the why and the how to overcome - since the forum pages seem to be impacted as a rule, and not an exception.  I appreciate the input though. 

Edited March 6, 2023 by brandoninsc

[Marc]

The only people who will really be able to answer that for certain would be google themselves unfortunately.