ResearchBib Share Your Research, Maximize Your Social Impacts
Sign for Notice Everyday Sign up >> Login

An Adaptive, Selective and Incremental Web Crawler

Journal: International Journal of Scientific Engineering and Research (IJSER) (Vol.3, No. 7)

Publication Date:

Authors : ; ;

Page : 198-202

Keywords : Web; web crawler; URL; Crawling policies; Incremental and Adaptive Crawler; Search Engine.;

Source : Downloadexternal Find it from : Google Scholarexternal

Abstract

The large size and the dynamic nature of the Web make it necessary to continually maintain Web based information retrieval systems. Crawlers facilitate this process by following hyperlinks in Web pages to automatically download new and updated Web pages.The expansion of the World Wide Web has led to a chaotic state where the users of the internet have to face and overcome the major problem of discovering information. For the solution to this problem, many mechanisms were created based on crawlers who are browsing the www and downloading pages. In this paper we will describe a crawling mechanism which is created in order to support data mining and processing systems and to obtain a history of the web?s content.

Last modified: 2021-07-08 15:25:04