Enhancement of 30-day acute care event prediction model following outpatient chemotherapy using vital signs, lab results, and advanced machine learning: A comparative analysis.

Abstract

e13586 Background: This study aimed to improve the predictive performance of acute care events within 30 days (ACE30) following outpatient chemotherapy administration. Previous work by Stein et al. (JOP, 2023), focused on initial chemo administration, utilized 9 demographic and therapeutic-related variables (e.g., age, race, BMI, cancer type, etc.), and achieved a positive predictive value (PPV) of 0.23 and area under the ROC curve (AUC) of 0.65. Our subsequent research (He et al., ASCO 2024) explored advanced machine learning (ML) models with resampling strategies using same variables, which yielded only minimal improvements in PPV and AUC. This study using all chemo administrations sought to enhance ACE30 prediction by incorporating additional vital signs and laboratory measurements into the existing predictor set. Methods: Data from 190,629 chemotherapy administrations delivered at Orlando Health between February 2012 to April 2021 were divided into training (133,440) and validation (57,189) sets. Two experiments were conducted: Experiment I incorporated pre-chemotherapy vital signs (e.g., temperature, blood pressure, and heart rate) with previous predictors. Experiment II added pre-chemotherapy laboratory measurements (WBC, HGB, Na, Ca, K, and Mg), and previous predictors. Three modeling approaches were employed: L1-penalized logistic regression, XGBoost (a nonlinear tree-based algorithm), and artificial neural networks (ANN). Using the output predicted probabilities, the top 10% patients were identified as high risk, and the remaining were identified as low risk. Results: In the validation dataset, the ANN model demonstrated superior performance in Experiment I, achieving a PPV of 0.62 & AUC of 0.79. Experiment II yielded comparable results with the ANN model (PPV: 0.56 & AUC: 0.78). Of the 154 administrations identified as high risk in Experiment I, 96 (62%) had an ACE30. Conclusions: The model expansion significantly improved the predictive performance of ACE30 events following outpatient chemotherapy administration, with ANN models consistently outperforming other modelling approaches. In our prior models, administrations identified as high risk had an actual ACE30 only 23% of the time. The ANN model with additional vital signs and labs developed in this study identified actual acute care events 62% of the time. These findings suggest pre-chemotherapy vital signs and possibly pre-chemotherapy lab results are crucial predictors for acute care events. These results are potentially strong enough to suggest implementation into our clinical workflows to lower rate of emergency room visits and hospitalizations.