Word replacement recognition using Sentence Oddity, K gram and NGD

ABSTRACT There are so many vigilance system which works on recognition of replaced word from sentence, for example in antiterrorism organizations like ATS. While sending messages incendiary changes the word which may cause to set alarm. For example, "we are ready for the blast tonight" can be transform into "we are ready for the complex tonight". This type of transformations or replacements can be happen and human being can easily recognized it. But for large documents or set of documents, emails, chat messages it is not possible to detect such transformations. To solve this issue we can make this process automatic with the help of frequencies of each word. For example in above sentence "complex" doesn't make any sense. So frequency of complex is less as compare to blast. We define three measures to detect the transformation or replacement of the word, Sentence Oddity (SO), K gram and Normalized Google Distance (NGD). With the help of these three algorithms we show that after combination of the three we can get 90% positive results. We also developed the watchlist which contains different words which may be replaced. After detecting the replaced word we are matching detected word with words from the watchlist. For this matching we used cosine similarity.