Data Cube Materialization with MR Cube and CM Sketch Approach

Data cube computations plays an important role in data warehouse systems. Applications with multidimensional data analysis are looking for unusual patterns. Here aggregation of data is done across many dimensions. Aggregation is done by making use of SQL aggregate functions and Group by operators. As there is need for multidimensional generalization of these operators, data cube is used which is a way for structuring data in multidimensions so that analysis can be done on some measures of interest. One of the key tasks in data warehouse is data cube computations. Several techniques for data cube computations are available but there are some limitations so MapReduce based approach can be used to overcome the limitations. MR-Cube, which is Mapreduced based approach creates lattices using derived data set which are further partitioned using value partitioning techniques followed by batch areas creation, makes an effective distribution of data and computation workload. Data cube computations in parallel using partially algebraic measures is done using MapReduced based algorithm. Extreme data skew is detected for a few cube groups that are unusually large. CM-Sketch is a Count Min Sketch approach, which is a compressed counting data structures used as a solution for extreme data skews.