FORMALIZING TOP K QUERIES FOR THE DOCUMENT STREAMS

In the present days, the accumulation of the information is of high that leads to the innovation of novel field, named, Big Data. Most of the online monitoring applications that exhibits stream of data such as call records, sensor readings, web usage logs, and network packet traces etc are to be preprocessed effectively for future aspects. Though, data has been emerged from different sectors, it has to be interpreted and stored in an effective way. A prior scheme suggests frequency and indexing order based data arrangements process that explores complex task in the view of big data systems. This paper focuses on designing a novel data arrangements and interpretation model for the data streams. We propose a tokenized top k query handling model which reduces the recomputational and memory utilization. Each data object is represented by frequency, identity and the lifespan of the objects. Relied upon the lifespan of the objects, its subsets are predicted and then formulated to the top k process. These top k objects are continuously analyzed and then declared as the ‘tokenized top k set'. By doing so, we can efficiently achieve the less memory consumption with an effective preprocessed data objects. Experimental analysis have been studied in the synthetic IBM T10I4D100K dataset in the terms of no. of generated candidate sets, time taken for first scan, time taken for second scan and the memory usage. It is compared with the existing, frequency ordering approach which proves that our proposed model achieves better results in memory usage and recomputational tasks.