An Optimized Approach for Processing Small Files in HDFS

In Todays world cloud storage, has become an important part of the cloud computing system. Hadoop is an open-source software for computing huge number of data sets to facilitate storage, analyze, manage and access functionality in distributed systems across huge number of systems. Many of the user created data are of small files. HDFS is a distributed file system that manages the file processing across huge number of machines in distributed systems with minimum hardware requirement for computation. The performance of the HDFS degrades when it is handling the storage and access functionality of huge number of small files. This paper introduces the optimized strategies to handle small file processing in terms of storage and access efficiencies. Replication algorithms HAR and sequenceFile, merging algorithms, replica placement algorithms, Structurally-Related Small Files (SSF) - File Merging and Prefetching Scheme (FMP) and SSF-FMP with three level prefetching-catching technology. The proposed strategies help in effective increase of access and storage efficiency of small files. Inclemently shorten the time spent for reading and writing of small files when requested by clients.