Wordpress Themes - WP Forum at BFA
Click here to register or to donate.
Auto self-registration is not available here - far too many spammers. This forum has many, many backlinks and because of that there is an intense desire among spammers to drop their links here.

Wordpress Themes - WP Forum at BFA » WordPress Themes » Atahualpa 3 Wordpress theme » Header configuration & styling »

robots.txt file is blocking Googlebot


  #1  
Old Apr 11, 2013, 03:04 AM
Ken Driver
 
9 posts · Aug 2012
Not sure this belongs in 'header' threads but could not see a more suitable category...
Keep getting an auto-generated email from Google that their googlebot cannot access the site. Google says the issue is the robots.txt file.
hoxtonindustries.com.au/robots.txt
I did not knowingly put this file on the server, nor did the client's SEO consultant, so assume it was auto-generated by either the Atahualpa theme, or by WordPress itself?
Here is the text that WAS in the file:
User-agent: *
Disallow: /_mm/
Disallow: /_notes/
Disallow: /_baks/
Disallow: /MMWIP/

User-agent: googlebot
Disallow: *.csi
The client's SEO consultant has since removed the last two lines, but still I get Google's 'access error' message. How did the file get there? Should I just delete it? What do /MMWIP/ and *.csi mean?

Also, in the Atahualpa Theme Options, there is some Google Analytics code put there by the SEO consultant. Could this code possibly be conflicting with the theme's own SEO Options? Not sure if this is in any way related to the robots.txt file (?) but I can turn off the theme SEO Options if you think that is advisable.
  #2  
Old Apr 11, 2013, 04:23 AM
juggledad's Avatar
juggledad
 
23,682 posts · Mar 2009
OSX 10.11.5 WP 4.x Atahualpa(all) Safari, Firefox, Chrome
1) Atahualpa does not create a robots.txt file
2) Wordpress does not creates a actual robots.txt file
3) check dashboard->settings->reading->Search Engine Visibility
One of the first steps in debugging any issue is 'turn off all plugins'

Quote:
I can turn off the theme SEO Options
sure, try it ank see what happens

Bookmarks

Tags
googlebot, robots

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] tag, category pages are showing noindex,follow for robots meta tag Schafir Atahualpa 3 Wordpress theme 3 Nov 15, 2011 05:44 PM
[SOLVED] Importing Atahualpa settings file, file not valid Neut New Versions, & Updating 12 Feb 1, 2011 08:35 AM
[SOLVED] Where to insert robots meta tags pyoterguru Header configuration & styling 5 Apr 2, 2010 07:18 AM
[SOLVED] robots: can't turn off noindex, nofollow dickeson Atahualpa 3 Wordpress theme 2 Feb 28, 2010 04:54 PM
[SOLVED] Lijit Search, robots.txt, and /Disallow Wimbledon Plugins & Atahualpa 2 Oct 3, 2009 09:24 PM


All times are GMT -6. The time now is 02:08 PM.


Powered by vBulletin® Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.