What is a Search Engine and How Does it works?

A search engine is a program that searches for sites on vast expanse of World Wide Web based on the words or phrases that you choose as search terms. Search engines look around their own databases of information in order to find the information that you are looking for.

Internet search engines are the web search engines that searches and retrieves information on the web. Most of them use crawler indexer architecture that means they depend on their crawler modules. Crawlers also called as spiders. Spiders are the small programs that browse the web.

Googlebot / Robot / Spider
Googlebot is a web crawling robot which finds and retrieves pages on the web and hand over to Google indexer. Googlebot uses an algorithmic process, computer programs to determine which site to crawl, how often and how many pages to fetch from each site. Google crawl process begins with a list of web page URL’s, generated from previous crawl processes and augmented with sitemap data provided by webmaster.

At the time of crawling Googlebot detects the links on each page and add them to its list of pages to crawl. New sites, changes to existing site and deadlines are noted and used to update the Google Index.

Google Indexer
To process the pages crawled by Googlebot Google uses indexing algorithm to organize crawled content. The indexer extracts the words form each page crawled by Googlebot and records its URLs. After processing this information it results into a large lookup table that gives a list of URLs pointing to pages where each word occurs. (Google processes information in key content tags for e.g. title tag or alternative tags of images.)

When a user enters a search query in Google Search box, Google machines search the index for matching pages (for entered keyword) and returns the pages that are most relevant to that search term.