GRACE: A conversational AI platform for geriatric oncology risk and capability evaluation.

Abstract

1658 Background: Comprehensive geriatric assessment (CGA) is a multidimensional evaluation of health in older adults and endorsed by the American Society of Clinical Oncology (ASCO) and the National Comprehensive Cancer Network (NCCN) for improved outcomes in geriatric cancer patients. However, CGA is underutilized due to time constraints and staffing shortages. Artificial intelligence (AI) can provide autonomous geriatric assessments (GA) but require validation. Methods: GRACE, an AI platform for GA via natural language interaction, was tested against manual GA. 70 adults aged ≥65 were enrolled from UCLA using the portal MyChart and at affiliated active senior living centers. 15 clinicians contributed qualitative perspectives. Concordance was evaluated using sensitivity, specificity, predictive values, and agreement (Cohen’s κ, Gwet’s AC1). Usability was measured with the System Usability Scale (SUS), and patient and provider interviews were thematically analyzed. Results: GRACE showed high diagnostic performance (sensitivity 0.73, specificity 0.94, κ=0.63, AC1=0.82) with highest agreement in polypharmacy (κ≈0.97; AC1≈0.98). Usability was high (SUS >80th percentile) and 84% of participants reported confidence using GRACE. Clinicians endorsed GRACE as a practical pre-visit intake tool that integrates with electronic health records and facilitates timely referrals. Conclusions: GRACE demonstrates that conversational AI can replicate key elements of clinician-administered GA while reducing burden on oncology teams. Larger multi-site studies are warranted to evaluate clinical impact and integration into routine oncology practice. Performance of GRACE versus manual geriatric assessment. Domain Sensitivity (CI) Specificity (CI) PPV (CI) NPV (CI) Cohen's Kappa (CI) Gwet's AC1 (CI) General Health 0.872 (0.748, 0.940) 0.931 (0.891, 0.957) 0.719 (0.592, 0.819) 0.973 (0.943, 0.988) 0.741 (0.637, 0.845) 0.887 (0.839, 0.935) Physical 0.859 (0.760, 0.922) 0.971 (0.956, 0.981) 0.753 (0.649, 0.834) 0.985 (0.973, 0.992) 0.781 (0.704, 0.858) 0.953 (0.935, 0.970) Functional 0.563 (0.332, 0.769) 0.985 (0.971, 0.993) 0.529 (0.310, 0.738) 0.987 (0.974, 0.994) 0.532 (0.298, 0.765) 0.972 (0.957, 0.986) Social Support 0.629 (0.530, 0.718) 0.873 (0.839, 0.900) 0.508 (0.420, 0.596) 0.918 (0.889, 0.940) 0.458 (0.359, 0.558) 0.753 (0.702, 0.805) Psychological 0.570 (0.460, 0.673) 0.920 (0.894, 0.940) 0.506 (0.404, 0.607) 0.937 (0.913, 0.955) 0.465 (0.353, 0.576) 0.839 (0.802, 0.876) Comorbidity 0.641 (0.484, 0.773) 0.996 (0.989, 0.998) 0.862 (0.694, 0.945) 0.985 (0.975, 0.991) 0.726 (0.601, 0.851) 0.980 (0.971, 0.989) Polypharmacy 0.974 (0.865, 0.995) 1.000 (0.893, 1.000) 1.000 (0.906, 1.000) 0.970 (0.847, 0.995) 0.971 (0.915, 1.027) 0.972 (0.916, 1.027) Overall 0.721 (0.674, 0.763) 0.956 (0.949, 0.963) 0.649 (0.603, 0.692) 0.968 (0.962, 0.974) 0.645 (0.604, 0.687) 0.917 (0.907, 0.927)