Detection and score grading for prostate adenocarcinoma using semantic segmentation

Abstract

Prostate cancer is a major global health challenge. In this study, we present an approach for the detection and grading of prostate cancer through the semantic segmentation of adenocarcinoma tissues, specifically focusing on distinguishing between Gleason patterns 3 and 4. Our method leverages deep learning techniques to improve diagnostic accuracy and enhance patient treatment strategies. We developed a new dataset comprising 100 digitized whole-slide images of prostate needle core biopsy specimens, which are publicly available for research purposes. Our proposed model integrates dilated attention mechanisms and a residual convolutional U-Net architecture to enhance the richness of feature representations. Class imbalance is addressed using pixel expansion and class weights, and a five-fold cross-validation method ensures robust training and validation. In model ensemble evaluation, the model achieves an average Dice of 0.87 and accuracy of 0.92 on the cross-validation held-out folds. When applied to completely unseen, external test data, the model demonstrates an average Dice of 0.64 and accuracy of 0.81. Segmentation and grading results were validated by a team of expert pathologists. Based on experimental results, this study demonstrates the potential of our proposed method and model as a valuable tool for the detection and grading of prostate cancer in clinical settings.