search engine optimization — basics

Keywords, Crawling and Indexing


In order to optimize your information for search, you need to understand how people search.The average number of words people type into the search field is 3.1. Those three words are key to users finding your content. These keywords are the way you connect with your audience.

Google takes the input of those three words and returns—frequently thousands, sometimes millions—of search results. Only 10 of these results appear on the first SERP. 80% of the users click one of the first three results.

Through direct observation of many billion searches and constant research and testing Google has a highly refined formula for delivering the best search results based on the three words entered by the user. Google's formula for search relevancy is based on more than 200 factors. The exact formula for displaying search results is Google's "Secret Sauce" and is not published. Google does provide extensive guidance that will help you make it your content "findable."

This tutorial focuses on Google's recommendations. Google had 64.9% of the search market in September 2009. The techniques presented here will also improve your "findability" in other search engines.

If you are publishing information on the web and want to reach the audience that would be interested in your content, Google Webmaster Tools is a primary resource. All of the tools and resources at the Google Webmaster site are free.


Google uses a "huge set" of computers to crawl billions of web pages. The program that does the crawling is called the Googlebot. Other search engines have similar programs generically called bots, robots or spiders.

The Googlebot uses web page URLs generated from previous crawls and new links found in the current crawl. One of the easiest and best ways to get your site found and thoroughly indexed is to provide a Sitemap. 

Sitemaps do not directly impact your place in search results. Sitemaps put information about how to search your site and what has changed on your site in one place. Microsoft and Yahoo! both officially support the Open Sitemap standard.

Google does not provide a tool for generating sitemaps. One of the free (for sites with fewer than 500 pages) tools for generating sitemaps is at


The googlebot reads and completely analyzes the content of your web site. The number and location of keywords, the relationship between words, links, images and image titles and captions all have an impact. This index of your site and the analysis of links coming into your site are the major factors in where your site appears in the natural search results on the SERP.