ResearchBib Share Your Research, Maximize Your Social Impacts
Sign for Notice Everyday Sign up >> Login

Topical Web Crawling for Domain-Specific Resource Discovery Enhanced by Selectively using Link-Context

Journal: The International Arab Journal of Information Technology (Vol.12, No. 2)

Publication Date:

Authors : ; ; ;

Page : 195-203

Keywords : Topical crawling; domain-specific resource retrieving; selectively using link context; DOM tree.;

Source : Downloadexternal Find it from : Google Scholarexternal

Abstract

To enable topical web crawling, link-context is the critical contextual information of anchor text for retrieving domain-specific resources. While some link-contexts may misguide topical web crawling and extract wrong web pages, because several relevant anchor texts become irrelevant or several irrelevant anchor texts become relevant after calculating the relevance between the link-contexts and the feature terms of the specific topic. In view of above, this paper presents a heuristic-based approach by selectively using link-context and implements DOM tree to locate the anchor text. Unlike previous crawling algorithms, which only zero in on link-context and ignore whether it is really needed or not. Our method cares both link-context and evaluating its necessity to correctly use link-context to guide topical crawling. Accordingly, our topical crawler can retrieve more relevant web pages. Experimental results indicate that this approach outperforms breadthfirst, best-first, anchor text only, link-context both in harvest rate and target recall

Last modified: 2019-11-14 22:29:36