Cookbook on Uncertainty Quantification in Medical Image Analysis

A collection of useful references to help you start your journey with uncertainty quantification (UQ) in medical image analysis (MIA).

We are continuing the work on updating the cookbook and your pull requests will be highly appreciated !

Other cookbooks

• Awesome uncertainty in Deep Learning [GitHub]
• UQ in DL [GitHub]
• Mathy UQ in DL [ArXiv 2024]
• Awesome Conformal Prediction [GitHub]

Tutorials

• UQ in MIA at MICCAI 2023 and 2024 [GitHub]
• UQ in ML for Engineering design and medical prognostics: A Tutorial [GitHuB]
• Calibrated uncertainty for regression [GutHub]

Software

• Uncertainty Toolbox [GitHub]

Reviews and surveys

General scope

(Source: Awesome uncertainty in Deep Learning)

• Gawlikowski et al. A survey of uncertainty in deep neural networks [Artificial Intelligence Review 2023] - [GitHub]
• Zhou et al. A survey on epistemic (model) uncertainty in supervised learning: Recent advances and applications. [Neurocomputing 2022]
• Abdar et al. A review of uncertainty quantification in deep learning: Techniques, applications and challenges [Information Fusion 2021]
• Hüllermeier & Willem Waegeman. Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods [Machine Learning 2021]

Medical image analysis

(Source: Google Scholar)

• Huang et al. A review of uncertainty quantification in medical image analysis: Probabilistic and non-probabilistic methods. [Medical Image Analysis 2024]
• Lambert et al. Trustworthy clinical AI solutions: A unified review of uncertainty quantification in Deep Learning models for medical image analysis. [Artificial Intelligence in Medicine 2024]
• Zou et al. A review of uncertainty estimation and its application in medical imaging. [Meta-Radiology 2023]
• Kurz et al. Uncertainty Estimation in Medical Image Classification: Systematic Review. [JMIR Medical Informatics 2022]

Healthcare

(Source: Other review papers, Google Scholar)

• Seoni et al. Application of uncertainty quantification to artificial intelligence in healthcare: A review of last decade (2013–2023). [Computers in Biology and Medicine 2023]
• Loftus et al. Uncertainty-aware deep learning in healthcare: A scoping review. [PLOS Digit Health 2022]
• Broekhuizen et al. A Review and Classification of Approaches for Dealing with Uncertainty in Multi-Criteria Decision Analysis for Healthcare Decisions. [PharmacoEconomics 2015]

Out-of-distribution detection

(Source: Google Scholar)

• Hong et al. Out-of-distribution Detection in Medical Image Analysis: A survey. [ArXiv]

Active learning

(Source: Google Scholar)

• Wang et al. A comprehensive survey on deep active learning in medical image analysis. [Medical Image Analysis 2024]
• Budd et al. A survey on active learning and human-in-the-loop deep learning for medical image analysis. [Medical Image Analysis 2021]

Applications

Separated by tasks (I. Classification, II. Segmentation, III. Regression) and by topics.

I. Classification

General

(Source: Google Scholar)

• Kurz et al. Uncertainty Estimation in Medical Image Classification: Systematic Review. [JMIR Medical Informatics 2022]

Out-of-distribution detection

(Source: Google Scholar)

• Linmans et al. Predictive uncertainty estimation for out-of-distribution detection in digital pathology. [Medical Image Analysis 2023]

II. Segmentation

General

(Source: Google Scholar)

• Zepf. Aleatoric and Epistemic Uncertainty in Image Segmentation. [PhD Thesis]

Quality Control

(Source: Lambert et al., 2024)

• Gonzalez et al. Distance-based detection of out-of-distribution silent failures for covid-19 lung lesion segmentation. [Medical Image Analysis 2022]
• Jungo et al. Analyzing the quality and challenges of uncertainty estimations for brain tumor segmentation. [Frontiers Neuroscience 2020]
• Roy et al. Bayesian quicknat: Model uncertainty in deep whole-brain segmentation for structure-wise quality control. [NeuroImage 2019]
• Graham et al. Mild-net: Minimal information loss dilated network for gland instance segmentation in colon histology images. [Medical Image Analysis 2019]
• Wang et al. Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. [Neurocomputing 2019]
• McClure et al. Knowing What You Know in Brain Segmentation Using Bayesian Deep Neural Networks [Frontiers Neuroinform 2019]

Inter-Rater Variability Modelling

(Source: Lambert et al., 2024)

• Baumgartner et al. PHiSeg: Capturing Uncertainty in Medical Image Segmentation. [MICCAI 2019]
• Hu et al. Supervised Uncertainty Quantification for Segmentation with Multiple Annotations. [MICCAI 2019]
• Jungo et al. On the Effect of Inter-observer Variability for a Reliable Estimation of Uncertainty of Medical Image Segmentation. [MICCAI 2018]
• Kohl et al. A Probabilistic U-Net for Segmentation of Ambiguous Images [NeurIPS 2018]

Human evaluation

• Huet-Dastarac et al. Quantifying and visualising uncertainty in deep learning-based segmentation for radiation therapy treatment planning: What do radiation oncologists and therapists want? [Radiother. Oncol.]
• Evans et al. The explainability paradox: Challenges for xAI in digital pathology. [Future Genertion Computer Systems 2022]

Domain Adaptation

• Xia et al. Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation. [Medical Image Analysis 2020]

III. Regression

Super-Resolution Reconstuction

(Source: Google Scholar)

• Angelopoulos et al. Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging. [ICML 2022]
• Qin et al. Super-Resolved q-Space deep learning with uncertainty quantification. [Medical Image Analysis 2021]
• Tanno et al. Bayesian Image Quality Transfer. [MICCAI 2016]