Robots.txt Protocol Enhanced by Big Search Engine Companies

I learned today, via Search Tools, that Microsoft, Yahoo! and Google have agreed to specific extensions to the robots.txt file protocol. All of their search engines will now honor additional directives. More info from Yahoo! and Google.

What is robots.txt some of you may be asking? It is a simple text file you can place on your web site to tell search engine spiders what parts of your site they should index and which they should ignore. It has been around for a long time and these are the first additions to the standard in at least a decade.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s