Wordpress Themes - WP Forum at BFA
There will be no more development for Atahualpa (or any other theme), and no support. Also no new registrations. I turned off the donation system. I may turn the forum to read only if it gets abused for spam. Unfortunately I have no time for the forum or the themes. Thanks a lot to the people who helped in all these years, especially Larry and of course: Paul. Take care and stay healthy -- Flynn, Atahualpa developer, Sep 2021

Wordpress Themes - WP Forum at BFA » WordPress Themes » Atahualpa 3 Wordpress theme » Plugins & Atahualpa »

[SOLVED] Lijit Search, robots.txt, and /Disallow


  #1  
Old Sep 30, 2009, 10:23 PM
Wimbledon's Avatar
Wimbledon
 
320 posts · Apr 2009
[VA & AZ], USA
Hello,

I implement Lijit Search on all my sites, as I really like the control it gives me over the search box and the detailed analytics.

I recently received a message from Lijit support (whom are great people, btw) stating this:

Quote:
It looks like you currently have a setting prohibiting crawlers from indexing your content. In order for us to get you properly crawled, you will need to edit the robots.txt file on your site and remove the following line:

Disallow: /

Until then, we won’t be able to index anything.
I have noticed that my Atahualpa sites were having some problems with content being indexed into my search engine, and according to Lijit, this is the problem.

I googled to learn more about the "Disallow" and want to know --- should I Disallow certain folders or can I just delete this line?

I am still relatively new to understanding the code and want to be sure that I know what I'm doing so I don't negatively impact how Google search crawlers see the site, etc.

Anyone else using Lijit?
  #2  
Old Oct 1, 2009, 04:25 AM
juggledad's Avatar
juggledad
 
23,765 posts · Mar 2009
OSX 10.11.5 WP 4.x Atahualpa(all) Safari, Firefox, Chrome
Atahualpa doesn't have a robot.txt, but you might have set one up with WP. See this article about Search Engine Optimization for WordPress.

http://codex.wordpress.org/Search_En...t_Optimization

At the bottom is a section on Robots.txt Optimization
__________________
"Tell me and I forget, teach me and I may remember, involve me and I learn." - Benjamin Franklin
Juggledad | Forum Moderator/Support
  #3  
Old Oct 3, 2009, 08:24 PM
Wimbledon's Avatar
Wimbledon
 
320 posts · Apr 2009
[VA & AZ], USA
Thanks for the helpful link.

What I realized now is that the robots.txt file is from the MyBB forum installation I paired with Atahualpa for this particular site.

Sorry for thinking this was an Atahualpa question...

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes


Similar Threads
Thread Thread Starter Forum Replies Last Post
Search Help joestyer Header configuration & styling 2 Oct 13, 2009 08:47 AM
How can I make the search wigget search in pages? aquilaciencia Plugins & Atahualpa 3 Aug 17, 2009 09:21 AM
Search box mhp Header configuration & styling 0 Jul 28, 2009 05:01 PM
Add Google Search to Search box? ldaily Header configuration & styling 0 Jul 21, 2009 09:12 AM
search box and archive search box zizi Page & Category Menu Bars 2 Mar 28, 2009 11:31 AM


All times are GMT -6. The time now is 02:28 PM.


Powered by vBulletin® Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.