LOAD BALANCING AND RUNTIME PREDICTION USING MAP REDUCE FRAMEWORK

MapReduce is a programming and execution framework for processing dataintensive applications on large cluster of computers and can be deployed on cloud computing environment. The ability to estimate and enhance the Map-reduce performance is needed to guide the scheduler for efficient resource scheduling and workload management. This paper presents an enhanced MapReduce framework by reducing the execution time of the job using in-memory mapper and dynamic workload predictor to meet the high-level performance goals at an optimal time. The workload predictor estimates the possibility of job completion with the user specified time. If the user defined time is not attainable then an estimated time span for the execution of the Job will be provided. For a Map reduce job that needs to be completed within a certain time, the job profile is built from the past executions of the jobs or by executing a smaller block of the job and there by predicting the expected estimation time of the entire job. Thus, the workload predictor and in-memory mapper helps in splitting the workload by the map reduce framework in an efficient manner.