ResearchBib Share Your Research, Maximize Your Social Impacts
Sign for Notice Everyday Sign up >> Login

SmartCrawler: For Site Locating and Balancing

Journal: International Journal of Science and Research (IJSR) (Vol.5, No. 6)

Publication Date:

Authors : ; ;

Page : 1931-1935

Keywords : Ranker; Form Classifier; Deep web; Crawler; Adaptive learning;

Source : Downloadexternal Find it from : Google Scholarexternal

Abstract

The number of web pages in Internet are changed day by day. Therefore relevant information searching is difficult task. Most of this data is hidden behind query forms which is interface to unexplored databases containing high quality structured data. Existing search engine has drawback that they cannot access and index this hidden data of the Web, in fact it is very hard to derive the unseen info. For this, we propose a framework, namely SmartCrawler, just to gather all the web interfaces in depth. Initially that is site locating, in this step, center pages are searched using search engines which is used to avoid visiting a large number of pages. For achieving precise results for a focused crawl, SmartCrawler gives ranks to websites to prioritize highly relevant ones for a given topic. The second step, adaptive link-ranking gives fast in-site searching by searching most relevant links. In third step, i. e. Web Navigation, Space complexity problem of other crawler are managed. The required relevant links are stored in database. Parsing of web pages is done in web navigation step. To reduce time in visiting some highly related links in unseen web directories, we have designed a link tree data

Last modified: 2021-07-01 14:39:08