Search engine optimization - Best practice -Invisionpower 4.2

[hameedacpa]

Hello everybody

I moved to Invisionpower forums since few years ago and I used to use Vbulletin before this and I'm always facing issue in InvisionPower forums with the indexing in google

the problem can be from my side and I need the best practice to make one topic indexed in the google search engine within 3 to 7 days maximum as it was when using VBulletin forums

I will start describing what I have

Suppose my main domain is mydomain.com and I installed the forums and pages in folder "forums" so when somebody type mydomain.com it will forward to mydomain.com/forums

in the root in Public_HTML there is two files htaccess and robots

htaccess in the main public_html

RewriteEngine on
RewriteRule ^$ /forums? [L,R=301]

# suPHP_ConfigPath /home/user/public_html
# .htaccess main domain to subfolder redirect
# Copy and paste the following code into the .htaccess file
# in the public_html folder of your hosting account
# make the changes to the file according to the instructions.
 
# Use PHP 5.3
#AddType application/x-httpd-php53 .php 

# Do not change this line.

# Change yourdomain.com to be your main domain.

# Change 'subfolder' to be the folder you will use for your main domain.

RewriteCond %{REQUEST_URI} !^/forums/

# Dont change this line.

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d

# Change 'subfolder' to be the folder you will use for your main domain.

#RewriteCond %{HTTP_HOST} ^(www.)?mydomain.com$ [OR]
#RewriteRule ^(.*)$ /forums/$1 [L]

# Change yourdomain.com to be your main domain again.
# Change 'subfolder' to be the folder you will use for your main domain
# followed by / then the main file for your site, index.php, index.html, etc.

ErrorDocument 400 /errors/400.htm
ErrorDocument 401 /errors/401.htm
ErrorDocument 403 /errors/403.htm
ErrorDocument 404 /errors/404.htm
ErrorDocument 500 /errors/500.htm

SetOutputFilter DEFLATE

# Caching Settings
<IfModule mod_headers.c>
  <FilesMatch "\.(flv|gif|jpe?g|png|ico|swf)$">
     Header set Cache-Control "max-age=2592000"
  </FilesMatch>
  <FilesMatch "\.(js|css|pdf|txt)$">
     Header set Cache-Control "max-age=604800"
  </FilesMatch>
  <FilesMatch "\.htm(l)?$">
     Header set Cache-Control "max-age=43200"
  </FilesMatch>
</IfModule>

# Libwww-perl Access Test
# RewriteCond %{HTTP_USER_AGENT} libwww-perl.* 
# RewriteRule .* – [F,L]

# IP Canonicalization Test
# RewriteCond %{HTTP_HOST} ^162\.144\.93\.19
# RewriteCond %{HTTP_HOST} ^mydomain\.com [nc]
# RewriteRule (.*) http://www.mydomain.com/$1 [R=301,L]

FileETag None
<IfModule mod_deflate.c> 
<filesMatch ".(css|js|x?html?|php)$">
    SetOutputFilter DEFLATE
  </filesMatch>
</IfModule>
<IfModule mod_expires.c>
ExpiresActive On
ExpiresByType image/gif "access plus 1 month"
ExpiresByType image/jpg "access plus 1 month"
ExpiresByType image/jpeg "access plus 1 year"
ExpiresByType image/png "access plus 1 month"
ExpiresByType text/css "access plus 1 month"
ExpiresByType text/javascript "access plus 1 week"
ExpiresByType application/javascript "access plus 1 week"
ExpiresByType application/x-javascript "access plus 1 week"
ExpiresByType application/x-shockwave-flash "access plus 1 week"
ExpiresByType video/x-flv "access plus 1 week"
ExpiresByType application/pdf "access plus 1 week"
ExpiresByType text/html "access plus 1 month"
ExpiresByType text/xml "access plus 1 month"
ExpiresByType image/x-icon "access plus 1 year"
ExpiresByType application/font-woff "access plus 1 month"
ExpiresByType application/x-font-woff "access plus 1 month"
ExpiresByType font/woff "access plus 1 month"
ExpiresByType application/font-woff2 "access plus 1 month"
ExpiresByType application/vnd.ms-fontobject "access plus 1 month"
ExpiresByType application/x-font-ttf "access plus 1 month"
ExpiresByType font/opentype "access plus 1 month"
ExpiresByType image/svg+xml "access plus 1 month"
</IfModule>

# <IfModule mod_headers.c>
# Header set Content-Security-Policy "script-src 'self'"
# Header always set X-FRAME-OPTIONS "DENY"
# Header always set X-XSS-Protection "1; mode=block"
# Header always set X-Content-Type-Options "nosniff"
# </IfModule>

 

and the robots file in public_html as follow :

Sitemap: http://www.mydomain.com/forums/sitemap.php
User-agent: *
User-agent: Mediapartners-Google
Disallow:
Disallow: /forums/cpadmin/
Disallow: /forums/attachments/
Disallow: /forums/cache/
Disallow: /forums/converge_local/
Disallow: /forums/hooks/
Disallow: /forums/ips_kernel/
Disallow: /forums/retail/
Disallow: /forums/public/style_captcha/
Disallow: /forums/index.php?app=core&module=task
Disallow: /forums/index.php?app=forums&module=moderate&section=moderate
Disallow: /forums/index.php?app=forums&module=extras&section=forward
Disallow: /forums/index.php?app=members&module=chat
Disallow: /forums/index.php?&unlockUserAgent=1
Disallow: /forums/*&module=search
Allow:	  /forums/*&search_tags=
Disallow: /forums/*app=core&module=global&section=reputation
Disallow: /forums/*app=core&module=usercp
Disallow: /forums/*app=members&module=messaging
Disallow: /forums/*&p=
Disallow: /forums/*&pid=
Disallow: /forums/*&hl=
Disallow: /forums/*&start=
Disallow: /forums/*view__getnewpost$
Disallow: /forums/*view__getlastpost$
Disallow: /forums/*view__old$
Disallow: /forums/*view__new$
Disallow: /forums/*view__getfirst$
Disallow: /forums/*view__getprevious$
Disallow: /forums/*view__getnext$
Disallow: /forums/*view__getlast$
Disallow: /forums/*&view=getnewpost$
Disallow: /forums/*&view=getlastpost$
Disallow: /forums/*&view=old$
Disallow: /forums/*&view=new$
Disallow: /forums/*&view=getfirst$
Disallow: /forums/*&view=getprevious$
Disallow: /forums/*&view=getnext$
Disallow: /forums/*&view=getlast$
Disallow: /forums/*?s=
Disallow: /forums/*&s=
Disallow: /forums/index.php?app=core&module=global&section=login&do=deleteCookies
Disallow: /forums/index.php?app=forums&module=extras&section=rating
Disallow: /forums/index.php?app=forums&module=forums&section=markasread
Disallow: /forums/*&do=who
Disallow: /forums/*&section=dname

also there is additional htaccess file in the folder /public_html/forums

<IfModule mod_rewrite.c>
Options -MultiViews
RewriteEngine On

RewriteBase /forums/
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule \.(jpeg|jpg|gif|png)$ /forums/public/404.php [NC,L]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /forums/index.php [L]
</IfModule>

More over the above the following are the setting in InvisionPower Dashboard

in the following meta tags I didn't add the full urls of my forums

also in the following I'll add the google webmaster tools showing the result of indexing.

from the above the percentage of indexing is 12% from the total urls sent to google for indexing and this consider very poor also I noticed that there is some issue with the archived topics for indexing.

I need your help for this and i hope the above show all the situation that I'm facing

By the way I'm not a professional in web pages and please I don't want anybody telling me to hire it for this because I bought the IP forum software and its support to make my life easy

[opentype]

2 hours ago, hameedacpa said:

I need your help for this and i hope the above show all the situation that I'm facing

No it doesn’t help, because we don’t have any access which would allow us to check out what is going on. You don’t even allow us to test a single URL on your (probably public) website, since you went to great lengths to hide your site URL from all the screenshots. 
So you need to check out yourself why Google can’t reach a large percentage of your site URLs in the sitemap. The Search Console even gives you the URLs. You just need to test them as a guest. If you can replicate errors, you need to check out why they exist and then resolve it this way. We can’t do that with screenshots with blackened URLs. 

[hameedacpa]

Actually what support team told me is to remove the robots file because this practice was exist in 3.x versions and now in 4.x it doesn't required and this solve partially my problem with google as all warnings gone