HIGH PERFORMANCE SEQUENCE MINING OF BIG DATA USING HADOOP MAPREDUCE IN CLOUD

Abstract Text mining can handle unstructured data. The proposed work extracts text from a PDF document is converted to plain text format, then document is tokenized and serialized. Document clustering and categorization is done by finding similarities between documents stored in cloud. Similar documents are identified using Singular Value Decomposition (SVD) method in Latent Semantic Indexing (LSI). Then similar documents are grouped together as a cluster. A comparative study is done between LFS (Local File System) and HDFS (HADOOP DISTRIBUTED FILE SYSTEM) with respect to rapidity and dimensionality. The System has been evaluated on real-world documents and the results are tabulated.