ResearchBib Share Your Research, Maximize Your Social Impacts
Sign for Notice Everyday Sign up >> Login

Study of a Web Crawler

Journal: International Journal of Scientific Engineering and Research (IJSER) (Vol.3, No. 8)

Publication Date:

Authors : ; ;

Page : 14-16

Keywords : Web CrawlerURLSearch EngineCrawling Policies.;

Source : Downloadexternal Find it from : Google Scholarexternal


This is a study of the science and practice of web crawling. Web crawling may appear to be merely an application of breadth-first-search, the truth is that there are many challenges ranging from systems concerns such as managing very large data structures to theoretical questions such as how often to revisit evolving content sources.

Last modified: 2021-07-08 15:26:54