THE RELATIONSHIP BETWEEN USER PREFERENCES AND IR PERFORMANCE: EXPERIMENTAL USE OF BEHAVIORAL SCALES FOR GOAL ALIGNMENT IN IR PROJECTS

This paper tells the story of a series of experiments designed to explore the relationship between behavioral preferences and user performance in information retrieval projects. The experiments are a set of monitored user interactions with a randomly selected set of documents from a large corpus. Users’ behavioral preferences are recorded in a pre-test questionnaire, and their subsequent sessions are measured against standardized IR performance metrics of Recall and Precision. User IR performance is analyzed for significant correlations with a set of behavioral scales. The scales are designed to measure user preferences in the areas of tolerance for ambiguity, locus of control, innovativeness in technology, and dispositional innovativeness. Our findings support that a relationship exists between IR performance measures of recall and precision, and a user’s behavioral preferences. Our findings also suggest that behavioral preferences may be used to create a predictive model to forecast a user’s IR performance. These findings can be applied to organizations that prioritize strategies depending on the orientation of the searching and sorting goals for an electronic document collection being reviewed