Carolus H. J. Kusters's research while affiliated with Eindhoven University of Technology and other places

In endoscopy, imaging conditions are often challenging due to organ movement, user dependence, fluctuations in video quality and real-time processing, which pose requirements on the performance, robustness and complexity of computer-based analysis techniques. This paper poses the question whether Transformer-based architectures, which are capable to directly capture global contextual information, can handle the aforementioned endoscopic conditions and even outperform the established Convolutional Neural Networks (CNNs) for this task. To this end, we evaluate and compare clinically relevant performance and robustness of CNNs and Transformers for neoplasia detection in Barrett’s esophagus. We have selected several top performing CNN and Transformers on endoscopic benchmarks, which we have trained and validated on a total of 10,208 images (2,079 patients), and tested on a total of 4,661 images (743 patients), divided over a high-quality test set and three different robustness test sets. Our results show that Transformers generally perform better on classification and segmentation for the high-quality challenging test set, and show on-par or increased robustness to various clinically relevant input data variations, while requiring comparable model complexity. This robustness against challenging video-related conditions and equipment variations over the hospitals is an essential trait for adoption in clinical practice. The code is made publicly available at: https://github.com/BONS-AI-VCA-AMC/Endoscopy-CNNs-vs-Transformers.

Virtually all endoscopic AI models are developed with clean, high-quality imagery from expert centers, however, the clinical data quality is much more heterogeneous. Endoscopic image quality can degrade by e.g. poor lighting, motion blur, and image compression. This disparity between training, validation data, and real-world clinical practice can have a substantial impact on the performance of deep neural networks (DNNs), potentially resulting in clinically unreliable models. To address this issue and develop more reliable models for automated cancer detection, this study focuses on identifying the limitations of current DNNs. Specifically, we evaluate the performance of these models under clinically relevant and realistic image corruptions, as well as on a manually selected dataset that includes images with lower subjective quality. Our findings highlight the importance of understanding the impact of a decrease in image quality and the need to include robustness evaluation for DNNs used in endoscopy.