CLINICAL DECISION SUPPORT SYSTEM WITH PROCESSING OF MULTIMODAL MEDICAL DATA FOR RADIOLOGIST EFFICIENCY IMPROVEMENT PRACTICE

Subject of Research. The paper proposes a clinical decision support system using multimodal data. Method. The system is based on neural network models. The author proposes the methods of transfer learning with domain adaptation techniques, training on multimodal data and taking into account the negative knowledge transfer for the quality improvement of preliminary analysis of medical data with application of neural network models. Algorithms of neural networks for preliminary analysis of the patient study provide the analysis of multimodal data with consideration of the combination peculiarities of modalities, training of neural networks with specific domain adaptation techniques, and taking into account the phenomenon of the knowledge negative transfer for the increase of the neural network robustness to the natural equipment noise. Main Results. The proposed system is implemented as a web-platform. The implemented system effectiveness was assessed by involving 30 experts: doctors, students, and patients. A comparative assessment of the problems solution in various scenarios was carried out with the proposed system and without it. Experiments have shown that the proposed system application reduces the execution time of the most critical stages of scenarios. Practical Relevance. The proposed system is located in the ITMO domain at the uniform resource locator address: http://mcp.itmo. ru and is used in the practice of the Almazov National Medical Research Centre of the Ministry of Health of Russia, and the “City Children's Infectious Disease Clinical Hospital” Healthcare Institution in Minsk. The system is recommended for the application in clinical hospitals, individual practitioners and teaching radiologists.