Abstract Sat402: Variability in Expert Confidence when Neuroprognosticating after Cardiac Arrest

Abstract

Introduction: Neuroprognostication following cardiac arrest varies between centers and individual clinicians with inconsistent adherence to guideline-based strategies. Most in-hospital deaths occur after withdrawal of life-sustaining therapy for perceived poor prognosis. In addition to an actual prognostic estimate, expert confidence in their assessment may affect these clinical decisions and patient outcomes. Hypothesis: Variability in experts’ confidence in prognostication is substantial even after accounting for patient factors. Methods: We performed a secondary analysis of the Optimizing Recovery after Cardiac Arrest (ORCA) study. Briefly, we presented expert clinicians with clinical cases in a standardized format, gave them the ability to ask any clarifying questions, and independently recorded a neuroprognostication and confidence—the latter on a six-level Likert scale from “completely uncertain” to “completely certain.” We operationalized all available patient data (e.g.: neurological exam, EEG, neuroimaging, vasopressor doses, etc.) as patient-level factors and developed an ordinal regression (using a cumulative link mixed model) to predict confidence from patient-level fixed effects, treating expert as a random intercept. We used multiple imputation with chained equations to create 1000 complete data sets, then backward selected fixed effects to create a parsimonious model that consistently converged. Results: A total of 38 experts evaluated 1428 unique cases (median patient age 62 years (IQR 51-71), 40% female) and provided 4325 prognoses and confidences. Experts provided median 70 assessments (IQR 45-115), and the median confidence was “very certain” (IQR “somewhat certain”-“very certain”). After accounting for all patient factors, the median odds ratio summarizing between-expert confidence variability was 1.57. In a bootstrapped analysis, the standard deviation of inter-expert variance for assessments of the same case was 0.593 (+/- 0.083), whereas the standard deviation of inter-case variance for assessments by the same expert was 0.595 (+/- 0.059). Conclusions: A significant amount of the variability in confidence is explained by the tendencies of individual experts, rather than patient factors.