Multi-site blinded validation of a deep learning approach for clinical-grade MSI/dMMR detection in colorectal cancer from H&E-stained pathology images.

Abstract

44 Background: Testing for microsatellite instability (MSI) or mismatch repair deficiency (dMMR) is part of the diagnosis and clinical management of patients with colorectal cancer (CRC). Healthcare services recommend MSI or dMMR testing for all CRC patients to guide therapeutic choices and assist in identifying Lynch Syndrome. However, in clinical practice, high costs and the demand for timely test results, combined with the rising prevalence of CRC and a shrinking pathology workforce, present a barrier to universal adoption. This highlights the need for rapid and affordable alternatives. PANProfiler CRC (PPC) is a deep learning-based solution for detecting MSI/dMMR in CRC tumors that only requires whole slide images (WSIs) of haematoxylin and eosin (H&E)-stained tissue to provide test results. Using only WSIs, PPC offers an efficient alternative to standard testing. This study evaluates PPC's performance in a multi-site blinded setting. Methods: Blinded validation was performed using 3246 WSIs of H&E-stained CRC specimens. PPC provided outputs as "Stable", "Unstable", or "Indeterminate", with "Unstable" indicating dMMR or MSI-High, and "Stable" indicating proficient mismatch repair or non-MSI-High. "Indeterminate" was returned when PPC did not have a definitive result. PPC was evaluated by comparison to standard MSI/dMMR tests. Validation data spanned three cohorts from two sites (Table). St James’s University Hospital (SJUH), Leeds, UK, supplied Cohorts 1 and 2; Cohort 3 was sourced from Wales Cancer Biobank (WCB), UK. Blinded analysis was performed at SJUH. Results: Results are given (Table). PPC demonstrated an overall percent agreement of 93.91%, a positive percent agreement of 92.17%, and a negative percent agreement of 94.15%, returning a definitive result for 88.05% of WSIs. Conclusions: This real-world, multi-site, blinded validation study demonstrates PPC’s remarkable performance, comparable to standard tests for detecting MSI/dMMR in CRC, with high test replacement rates. In the clinical setting, PPC could significantly accelerate testing and enable timely delivery of stratified treatment plans. This accurate and cost-effective diagnostic solution promises to revolutionize MSI/dMMR testing in CRC. Blinded validation results of PPC with confidence intervals (CI) at 95%. Site Cohort Sample Size (Unstable; Stable) Overall Percent Agreement % (CI) Positive Percent Agreement % (CI) Negative Percent Agreement %(CI) Test Replacement Rate % SJUH 1 488 (78; 410) 92.79 (89.86-95.08) 90.16 (79.81-96.30) 93.24(90.11-95.62) 85.25 SJUH 2 2704 (318; 2386) 94.31(93.31-95.21) 92.31(88.48-95.18) 94.57(93.52-95.50) 88.42 WCB 3 54 (11; 43) 84.31 (71.41-92.98) 100.00(71.51-100.00) 80.00(64.35-90.95) 94.44 All 3246 (407; 2839) 93.91 (92.97-94.76) 92.17(88.82-94.78) 94.15(93.16-95.04) 88.05