Management of Large Scale Data

Storage management is important for overall performance of the system. Slow access time of disk storage is crucial for performance of large scale computer systems. Two important components of storage management are buffer cache and disk layout management. The buffer cache is used to store disk pages in memory to speed up the access to them. Buffer cache management algorithms require careful hand-tuning for good performance. A self-tuning algorithm automatically manages to clean the page activity in the buffer cache management algorithm by monitoring the I/O activities. Buffer cache management algorithm is used for global data structure and it is protected by lock. Lock can because contention is helpful in reducing the throughput of the multi-processing system. Currently used solution for eliminating lock contention will directly degrade hit ratio of the buffer cache will result in poor performance in the I/O bound. The new approach, multi-region cache eliminates the lock contention without affecting the hit ratio of buffer cache. Disk layout approach improve the disk I/O efficiency, called Overwrite, and is optimized for sequential me /so from a single file. A new disk layout approach, called HyLog. HyLog achieves performance comparable to the best of existing disk layout approaches in most cases.