ResearchBib Share Your Research, Maximize Your Social Impacts
Sign for Notice Everyday Sign up >> Login

DETERMINATION OF RESOURCE USAGE CHARACTERISTICS FOR HADOOP MAP REDUCE TASKS

Journal: JOURNAL OF COMPUTER ENGINEERING & TECHNOLOGY (JCET) (Vol.9, No. 1)

Publication Date:

Authors : ; ;

Page : 113-119

Keywords : Hadoop; Big data; map reduce scheduler; Resource Manager;

Source : Downloadexternal Find it from : Google Scholarexternal

Abstract

Hadoop is a common frame work used to process large amounts of data. It uses map reduce framework to divide the data and process it parallel on multiple nodes. Different jobs have different resource usages of CPU and IO and similarly different nodes have different loads. If resource usage of jobs and resource availability of nodes are considered in the decision of scheduling of multiple map and reduce tasks of different jobs, an optimized execution time can be obtained. It is more useful in could environment as map/reduce tasks execute on virtual machines in spite of physical machines. As parts of research conducted to build a dynamic scheduler for map reduce applications considering job and VM characteristics, this paper proposes a technique to study the job characteristic in terms of CPU and IO of usage

Last modified: 2018-09-17 16:00:09