Performance Analysis of K-Nearest Neighbour Classifier and Cosine Similarity Measure in Generating Weighted Scores for an Automated Essay Grading System

Grading of student academic performance in any examination is an inevitable exercise in educational assessment. This has become a major challenge in institutions where students? enrolment is enormous. Efficiency of the grading techniques determines the fate of the examinee. This study focuses on analyzing the performance of grading techniques used in automated essay grading system (AES): Cosine Similarity and K-Nearest Neighbor Classifier. These two techniques among others are used to compare documents and allocate similarity score, which is used to determine weighted score of a student. In this research, electronic copy of marking scheme(MS) and students? response(SR) were acquired in.txt format, preprocessed to remove stopwords, vector space model was used to derive the document vectors of MS and SR. The document vectors were compared using the cosine similarity measure and the k-nearest neighbor classifier to derive the similarity score. The machine generated student score was computed as a weighted aggregate of similarity scores, where the weight is the mark assigned for each question in the marking scheme. Performance evaluation was carried out on two datasets: CMP 401(Organization of Programming Languages) and CMP 201(Operating System) courses by comparing the effectiveness of the k-nearest neighbor classifier and cosine similarity measure on weighted scores using coefficient of determination. (R2). Results shows that Cosine Similarity Measure is more efficient in comparing document vectors for an automated essay grading system.