Topical Web Crawling for Domain-Specific Resource Discovery Enhanced by Selectively using Link-Context
Journal: The International Arab Journal of Information Technology (Vol.12, No. 2)Publication Date: 2015-03-01
Authors : Lu Liu; Tao Peng; Wanli Zuo;
Page : 195-203
Keywords : Topical crawling; domain-specific resource retrieving; selectively using link context; DOM tree.;
Abstract
To enable topical web crawling, link-context is the critical contextual information of anchor text for retrieving domain-specific resources. While some link-contexts may misguide topical web crawling and extract wrong web pages, because several relevant anchor texts become irrelevant or several irrelevant anchor texts become relevant after calculating the relevance between the link-contexts and the feature terms of the specific topic. In view of above, this paper presents a heuristic-based approach by selectively using link-context and implements DOM tree to locate the anchor text. Unlike previous crawling algorithms, which only zero in on link-context and ignore whether it is really needed or not. Our method cares both link-context and evaluating its necessity to correctly use link-context to guide topical crawling. Accordingly, our topical crawler can retrieve more relevant web pages. Experimental results indicate that this approach outperforms breadthfirst, best-first, anchor text only, link-context both in harvest rate and target recall
Other Latest Articles
- Arabic Text Classification using K-Nearest Neighbour Algorithm
- Optimized Features Selection using Hybrid PSOGA for Multi-View Gender Classification
- Optimizing Ontology Alignments by using NSGA-II
- The Proposal of a Qualification Based Approach to Teach Software Engineering Course
- Chaos Genetic Algorithm Instead Genetic Algorithm
Last modified: 2019-11-14 22:29:36