Evaluating real-world performance of an automated offline glaucoma AI on a smartphone fundus camera across glaucoma severity stages

Abstract

Purpose Leveraging an artificial intelligence system (AI) for glaucoma screening can mitigate the current challenges and provide prompt detection and management crucial in averting irreversible blindness. The study reports the real-world performance of a glaucoma AI system deployed on a smartphone-based fundus camera across various severities of glaucoma. Methods In this prospective comparative study at a tertiary care glaucoma clinic, consecutive patients were evaluated by a glaucoma specialist using clinical assessment, visual field tests, and SD-OCT, and categorized as definite glaucoma, glaucoma suspect, or no glaucoma. For glaucoma patients, severity was determined using Hoddap-Parrish-Anderson criteria based on visual field mean deviation (MD). A disc-centered image per eye was captured using a validated portable non-mydriatic fundus camera. The AI tool’s ability to detect referral-warranted glaucoma (glaucoma and glaucoma suspects) versus no glaucoma was compared to the specialist’s diagnosis. Results We included 213 participants with a mean age of 55 ± 14.7 years (18, 88). The glaucoma specialist diagnosed 129 subjects as definite glaucoma (early-23, moderate-31, severe-75), 33-disc suspects and 51 as no-glaucoma. The automated AI system based on fundus images achieved an overall diagnostic accuracy of 92.02%, sensitivity of 91.36% (95%CI 85.93% to 95.19%) and specificity of 94.12% (83.76% to 98.77%) for referral warranted glaucoma. The 14 false negatives included 5-disc suspects and 9 definite glaucoma (3-early, 3-moderate and 3-advanced glaucoma). The sensitivity of AI for detecting early, moderate and advanced glaucoma was 86.9% (95%CI 66.4–97.2), 90.3% (95%CI 74.3–97.96), and 96% (88.75% to 99.17%) respectively. Conclusion In a real-world setting, the AI-based offline tool integrated on a smartphone fundus camera showed a promising performance in detecting referral-warranted glaucoma compared to a glaucoma specialist’s diagnosis. The AI showed higher accuracy in detecting advanced glaucoma followed by moderate and early glaucoma.