Aiding genetic analysts: Design of a literature evaluation system

This paper is concerned with the design of a system that handles published research literature evaluation related to clinical DNA sequencing and analysis of genetic variants. The literature handling system is part of a larger system, the Norwegian clinical genetic Analysis Platform, currently under development at the Department of Medical Genetics at Oslo University Hospital. The genAP project has inquired into data handling requirements, procedures and supportive bioinformatics tools for analysis of genetic data. Finding and evaluating relevant literature that reports on clinical classifications of genetic variants is an important part of this process. In many cases, it is a requirement to compare local assessments with those published in high-quality external references, ensuring that the correct decision on the clinical nature of the variant is reached. The implications of the decisions made as part of this process are relevant for both patients and knowledge production and its transferability. We chose to use user-centered design as our research approach, in both qualitative (walk-troughs, interviews and talk-aloud evaluations) and quantitative (questionnaire) inquiries. User involvement in design and evaluation of the reference handling prototype was important for identifying diverse usability problems and design issues, which could then be improved in later iterations of the prototype. These issues included identifying the most relevant articles for a particular genetic variant and communicating uncertainty in individual assessments. Users have also contributed to defining more general guidelines for the re-design of later versions, e.g., a need for customization, as users often have different strategies for working with references. We assert that user involvement in the design and evaluation processes, such as described in this paper, leads to system design that is more in tune with users' needs, making the adoption and use of the system easier and improving the efficiency and quality of the analysis.