UNCERTAINTY ESTIMATION AND USAGE FOR DEEP LEARNING MODELS
Journal: International Scientific Journal "Internauka" (Vol.1, No. 99)Publication Date: 2020-12-15
Authors : Hvozdiev Valerii; Hvozdieva Tetiana; Strakhov Yevhen;
Page : 15-20
Keywords : machine learning; deep learning; uncertainty estimation; selective predictor; image classification;
Abstract
The default DL approaches in ML tend to output only prediction, but not an uncertainty measure alongside the prediction. There are several approaches to DL model modification that allow deciding if the model can be trusted. The approaches vary by computational load and performance considering given constraints. In the real world project, it is often not possible to modify the model or perform retraining to apply common uncertainty estimation techniques (black box problem). In the first part of the paper, we aim to measure the uncertainty of the model in practice. We have researched tolerable perturbations as a way to enforce noise in the input data. A framework was built that acts as a compound for a prediction model in image classification tasks and allows output uncertainty for given samples. For test purposes, a CNN model will be used over the CIFAR-10 dataset to showcase uncertainty evaluation. We also show how to use uncertainty values to get data insights into a real-world task In the second part, we discuss how to get a model to know when prediction is uncertain. We built a selective classifier to increase the performance of the model by narrowing the confidence interval on the input data and used the aforementioned uncertainty estimations in the rejection classifier. To showcase classifier features, we made an experiment with a softmax-based uncertainty classifier (vanilla) and Dirichlet distribution based value. To measure the performance of the predictor, we took the Brain Tumor Classification (MRI)[6] dataset as an example. For the received predictors we measured coverage and selective risks. We have shown that one could get significant accuracy gains by using selective models given accurate uncertainty measure.
Other Latest Articles
- THE ANALYSIS OF FOREIGN SCIENTIFIC DISCOURSE ON RESEARCH IN THE FIELD OF CIVIL PROTECTION AND MANAGEMENT IN EMERGENCIES
- THEORETICAL ASPECT OF PROSPECTS OF DEVELOPMENT OF THE SYSTEM OF PROFESSIONAL SELECTION OF FUTURE POLICE OFFICERS
- LAWYER IN THE FREE LEGAL AID SYSTEM IN THE EU COUNTRIES (ON THE EXAMPLE OF THE PRO BONO MODEL)
- STRATEGIC DIRECTIONS OF DEVELOPMENT OF ELECTRONIC COURT RECORD KEEPING IN THE CONTEXT OF THE CONCEPT OF DEVELOPMENT OF E-GOVERNMENT OF UKRAINE
- APPLICATION OF CIVIL LAW STANDARDS IN PROCUREMENT CONTRACTS
Last modified: 2021-04-20 17:40:21