Looking outside the box with a pathology aware AI approach for analyzing OCT retinal images in Stargardt disease

Abstract

Abstract Stargardt disease type 1 (STGD1) is a genetic disorder that leads to progressive vision loss, with no approved treatments currently available. The development of effective therapies faces the challenge of identifying appropriate outcome measures that accurately reflect treatment benefits. Optical Coherence Tomography (OCT) provides high-resolution retinal images, serving as a valuable tool for deriving potential outcome measures, such as retinal thickness. However, automated segmentation of OCT images, particularly in regions disrupted by degeneration, remains complex. In this study, we propose a deep learning-based approach that incorporates a pathology-aware loss function to segment retinal sublayers in OCT images from patients with STGD1. This method targets relatively unaffected regions for sublayer segmentation, ensuring accurate boundary delineation in areas with minimal disruption. In severely affected regions, identified by a box detection model, the total retina is segmented as a single layer to avoid errors. Our model significantly outperforms standard models, achieving an average Dice coefficient of $$99\%$$ for total retina and $$93\%$$ for retinal sublayers. The most substantial improvement was in the segmentation of the photoreceptor inner segment, with Dice coefficient increasing by $$25\%$$ . This approach provides a balance between granularity and reliability, making it suitable for clinical application in tracking disease progression and evaluating therapeutic efficacy.