Web Crawler: Essential Component of Search Engine
Journal: International Journal of Science and Research (IJSR) (Vol.4, No. 4)Publication Date: 2015-04-05
Authors : Akshada K. Dhakade; Deepak C. Dhanwani;
Page : 1471-1474
Keywords : Search Engine; Web Crawler; Crawler Policies; Techniques;
Abstract
With the vast growth of the Internet, many web pages are available online. Search engines use a component called as web crawlers for collecting these web pages from the web for storage and indexing. Many web pages are autonomous and are updated independent of the users. As the web pages are updated autonomously, users do not come to know of how often the sources change. Web crawler is the central part of the search engine which browses through the hyperlinks and stores the visited links for the future use. This paper represents concepts of web crawlers, its architecture and its various types.
Other Latest Articles
- The Maximum and Minimum Conditions in Fuzzy Soft Ideals of a Fuzzy Soft Lattice
- Analysis of Cognitive Radio a Way to Increase Spectrum Efficiency Using Periodogram
- The Role of the Cultural Institutions in the Civic Education
- Prevalence of Obesity among School Going Children in Selected Schools
- Non-Cpap Conservative Treatment Alternatives for Patients with Obstructive Sleep Apnea/Hypopnea Syndrome
Last modified: 2021-06-30 21:44:39