ResearchBib Share Your Research, Maximize Your Social Impacts
Sign for Notice Everyday Sign up >> Login

An Elegant Fusion of Concurrent Crawling and Page Rank Technique for Spidering Websites

Journal: International Journal of Computer Science and Mobile Computing - IJCSMC (Vol.3, No. 6)

Publication Date:

Authors : ; ;

Page : 247-255

Keywords : Web Crawler; Web Spider; concurrent crawler;

Source : Downloadexternal Find it from : Google Scholarexternal

Abstract

The World Wide Web is expanding day by day. With the great growth of the Web, it has become a massive challenge for the all-purpose single process crawlers (A crawler is a program that downloads and stores Web pages, often for a Web search engine) to locate the resources that are precise and relevant in an appropriate amount of time, so more enhanced and convincing algorithms are in stipulate. Thus it becomes vital to improve the crawling procedure, in order to finish downloading pages in a sensible amount of time. Web crawler which employs multi-processing to allow multiple crawler processes to run concurrently. We have proposed a resourceful concurrent crawler that is fusion of page rank and concurrent multi-process crawler, offering a means to efficiently crawl the Web and presenting a scalable solution that allows crawl speeds to be tuned as needed.

Last modified: 2014-06-18 01:32:32