Essay raters' personality types and rater reliability

There is a consensus among the scholars that rater reliability is of a critical importance in rating subjective tests like an essay. Raters’ variances in their judgments as the main threat to rater reliability have been attributed to different sources, all of which have the complex factor of rater individuality at their core. Based on this, it is argued in this study that essay raters’ personality types can influence the degree of intra-rater reliability in scoring compositions. Guideline presentation effect on reliability has also been measured. Thirty raters participated in the current study and the NEO-FFI was used to measure their personality types. The results indicated a degree of relationship between raters’ gender and personality types and their intra-rater reliabilities. However, no significant relationship was found between the guideline presentation and their inter-rater reliabilities.