Validation of metastatic breast cancer (mBC) diagnosis in structured electronic health record (EHR) data vs chart-curated verification using the IntegraConnect Precision-Q (IC PQ) de-identified database.

Abstract

e23147 Background: Leveraging EHR data to study cancer provides details from physician notes and scanned reports not available in claims data. The IC PQ De-Identified Database contains harmonized structured and unstructured EHR components from community and academic oncology practices across the US, including > 3 million patient (pt) records, some of which contain verified, curated data. Understanding the accuracy of structured data vs curated data is an important component of research and quality studies, specifically when leveraging only structured data. Methods: We compared structured data to chart-curated data (gold standard) to assess the accuracy of mBC pt identification. We identified mBC pts from structured data using ICD codes or TNM values and explored the accuracy of site-specific metastases (mets) for bone, brain, lung, and liver. We reported sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV), with a focus on sensitivity and specificity as PPV and NPV vary by disease prevalence. The data include curated pts with a BC diagnosis in the IC PQ deidentified database from 1-Jan-2014 to 30-Jun-2024. Results: We identified 10,048 records with a BC diagnosis that also contained curated information, among which 1778 pts had curation-confirmed mBC. Structured data identified 1700 records with mBC, of which 1558 records were true positives. The sensitivity of structured data for identifying mBC patients was 87.6% (95% confidence interval [CI] 86.0-89.1) with 98.3% (95% CI 98.0-98.6) specificity. Structured data missed 220 (12.4%) pts with mBC. Bone mets had the highest sensitivity, followed by brain mets. Liver and lung mets had sensitivities < 60% driven by high numbers of false negatives. Specificity was > 90% for all site-specific mets except bone. Conclusions: IC PQ structured data had high sensitivity and specificity. Its accuracy can be attributed to EHR systems providing comprehensive structured data beyond ICD codes. However, site-specific mets identified from ICD codes alone had lower sensitivity and specificity, and the ability of structured data to accurately identify pts with brain, lung, or liver mets is limited. Studies with an interest in site-specific mets should consider a curation component to avoid misclassification. Sensitivity % (95% CI) Specificity % (95% CI) PPV % (95% CI) NPV % (95% CI) Mets 87.6 (86.0-89.1) 98.3 (98.0-98.6) 91.6 (90.2-92.9) 97.4 (97.0-97.7) Bone 84.0 (81.7-86.1) 88.6 (85.9-91.0) 93.0 (91.3-94.5) 75.4 (72.2-78.5) Brain 68.9 (63.2-74.3) 96.8 (95.8-97.6) 80.2 (74.7-85.1) 94.3 (93.0-95.4) Lung 48.7 (44.4-53.0) 96.1 (94.9-97.1) 84.5 (80.0-88.4) 81.2 (79.1-83.2) Liver 53.4 (49.3-57.4) 96.8 (95.6-97.7) 89.3 (85.6-92.3) 80.5 (78.4-82.6)