SURVEY AND PROPOSED PARALLEL ARCHITECTURE FOR INLINE DATA DeDUPLICATION USING SHA-2 ALGORITHM

In this digital world of internet, data storage and server storage use is often and every home user, enterprises, several organizations are using email and online storage as a storing node. Online backup storage is an easy option for everyone to store digital data, files and other multimedia files. Due to huge requirement of data storage online it is found that so many users stored the same data repeatedly. This makes the storage servers loaded as well as more disk storage is required to save a large amount of same data. Due to the same reason the search operation takes more time to found a specific file and time complexity to give positive search result and acknowledgement is increased. This problem was overcome with a mechanism known as Data de-duplication. This process is used for removing duplicate data and to reduce redundancy at server node. Data de-duplication method is usually applied at the time of storing the data (Inline) or after storing the data (Post process) and it can be applied over primary data or secondary data. De-duplication process can be accomplished at File, Block or Byte level. In previous modifications of de-duplication algorithms it is found that some issues like time complexity, unbalanced load, hash collision etc. has occurred. In this thesis report we have studied previous and recent work on de-duplication and proposed a solution which is a Parallel architecture for inline data de-duplication which uses the Secure Hash Algorithm (SHA) 256 for performing data de-duplication task in order to overcome the issues of time complexity and to reduce hash collision. In this architecture write and delete operations are performed for efficiency and time evaluation. The time taken is much lesser for redundant data during write procedure. This decrement in time is a result of using de-duplication process. This architecture is useful for storage servers where a huge amount is stored every day and software industries always looks for new developments so that they can keep their storage systems up to date and free for efficient utilization of the server nodes