Comparing machine and deep learning models for pediatric anxiety classification using structured EHRs and area-based measures of health data

Abstract

Objective This retrospective, case-control study with internal validation evaluates the performance of machine learning (ML) and deep learning (DL) models in classifying pediatric patients at risk for anxiety disorders using structured electronic health records (EHRs) and area-based measures of health (ABMH). The aim is to enable proactive care by monitoring potential anxiety onset across developmental stages. Methods We trained a series of ML models (Logistic Regression, Decision Tree, Random Forest, K-Nearest Neighbors, XGBoost) and DL models (LSTM, GRU, RETAIN, Dipole) using structured EHR data from 30-day windows prior to diagnosis. Two datasets were used per age group: one with structured EHR data only, and another including both EHR and ABMH data. ML models were trained using short-term cross-sectional features, while DL models leveraged full longitudinal patient histories. Performance was assessed using AUROC, AUPRC, PPV, NPV, F1 score, and accuracy. Due to differences in input scope, model performance reflects both algorithmic and temporal design differences and is not intended as a direct comparison between ML and DL. Results ML models offered strong baseline performance, with XGBoost achieving AUROC scores of 0.817 (EHR) and 0.816 (EHR+ABMH) for 8-year-olds. Adding ABMH features did not significantly improve performance. DL models, particularly RETAIN and Dipole, achieved the highest AUROC values (e.g., Dipole: 0.853 with EHR, 0.857 with EHR+ABMH for 8-year-olds), outperforming other DL and ML models within their respective design constraints. Conclusion Both ML and DL models successfully identified likely anxiety onset using structured EHR data. DL models using longitudinal data achieved the highest performance, while XGBoost provided a robust ML baseline. The minimal impact of ABMH features highlights integration challenges, and performance variation across ages emphasizes the need for age-stratified modeling approaches.