Abstract

Deep learning-based medical image segmentation has revolutionized healthcare diagnostics. While the accuracy offered by these models is important, ensuring their practical implementation requires a comprehensive reliability assessment. Among the sources of uncertainty, inter-rater variability, which reflects natural disagreements among annotators, has been historically overlooked. Studying this variability can be a key factor in improving the robustness and reliability of models.
Basing our experiments on paraspinal muscle segmentation, which has significant value in studies related to low back pain, in this dissertation, we first proposed a novel multi-task TransUNet model to accurately segment paraspinal muscles while predicting inter- rater labeling variability visualized using a variance map of raters’ annotations. Benefiting from the transformer mechanism and convolution neural networks, our algorithm was shown to perform better or similar to the state-of-the-art methods while predicting and visualizing multi-rater annotation variance per muscle group in an intuitive manner. Subsequently, we studied the relationship between inter-rater variability and aleatoric/epistemic uncertainties, in the context of DL model architecture and label fusion methods. Specifically, we measured aleatoric and epistemic uncertainties using test-time augmentation, test-time dropout, and deep ensemble to explore their relationship with inter-rater variability. Furthermore, we compared UNet and TransUNet to study the impacts of Transformers on model uncertainty with two label fusion strategies. This thesis provides novel frameworks for visualizing, understanding and quantifying inter-rater variability to better inform relevant deployment and implementation of DL models for medical image segmentation.