• 800.221.0266
  • +1 707.569.5600
  • E-Myth en Español

Newsletter Archive

Prev | Newsletter Archive | Next

Website Tips: Block the Bots

Published: 09/04/2008

The E-Myth Insider

4 September 2008 Brought to you by E-Myth Worldwide

Dear E-Myth Reader,

We previously discussed optimizing your web pages for higher placement in search engine results. Now we describe how to exclude pages from results in Website Tips: Block the Bots.


Block the Bots

Robby the Robot in 'Forbidden Planet' (MGM)

Many websites have public pages they'd rather not see in web search results, e.g., coupons, expired offers, style sheets and pages not fully functional. Private data should always require login/password authentication, but any public page could be read by a search bot and added to an index. You can keep pages hidden from major search engines using a simple text file named robots.txt.


You can see any site's robots.txt (if it exists) by typing the web domain address plus /robots.txt. For instance http://www.google.com/robots.txt lists content directories that Google wants to exclude from search results. Do you have public pages you don't want broadcast by the big search engines? Read Website Tips: Block the Bots.


On The Website

Group Business Coaching

Join a small group of like-minded business owners to share experiences, evaluate best practices and acquire new business skills, all directed by an E-Myth Business Coach.

Group Mastery Impact™ leverages a group coaching environment, deep online content and a great price to help you take your business to the next level. See the Group Mastery Impact page for details.

Yours Truly,
The E-Myth Team


View Our Newsletter Archive

Subscribe to the E-Myth ViewPoint Newsletter

  • Valuable Articles
  • No Spam
  • Low Volume (one/week)