Cox models vs. machine learning for survival prediction: Do traditional approaches still hold their ground?

Abstract

e13647 Background: Sarculator, a Cox model-based nomogram, has been widely used for survival predictions in patients with extremity soft tissue sarcomas (eSTS), demonstrating user-friendliness and reliability. With growing interest in machine learning (ML), this study investigates whether, given the Sarculator prognostic variables, these more complex approaches offer meaningful improvements. Methods: Data from 3,748 patients with eSTS from four international cohorts were used, including the Sarculator’s development cohort (1,452 patients, Milan, Italy) and the three original external validation cohorts (Toronto, Canada; Villejuif, France; London, UK). Predictions were compared in terms of discrimination (C-index, the higher the better), and calibration (plots; 5- and 10-year Brier score, the lower the better. Four ML models — Extreme Gradient Boosting (XGBoost), Model-Based Boosting (MBoost), Random Survival Forests (RSF), and Optimal Survival Trees (OST)—were benchmarked against Sarculator, all including the same Sarculator covariates. A hybrid SuperLearner, which combined predictions from the Sarculator Cox model and the best-performing ML model, was also evaluated. Results: Sarculator Cox model consistently performed well across the four cohorts (C-index: 0.698–0.775) with reliable calibration and low Brier scores. ML models, particularly XGBoost, demonstrated slightly better calibration but poorer generalizability in external cohorts. MBoost and RSF exhibited calibration-discrimination trade-offs, while OST underperformed compared to all the other models. The SuperLearner, integrating predictions from the Cox and XGBoost models, marginally improved calibration but provided limited additional value compared to the Cox model. Conclusions: Sarculator’s robust performance across development and validation cohorts highlights that traditional Cox models remain clinically valuable. The added complexity of ML approaches does not necessarily result in superior prediction accuracy. Importantly, Cox models retain their interpretability, essential for clinical application, whereas ML models required complex tools for explanation. In the context of clinical-based variables, clinicians might be more likely to prioritize models offering simplicity and reliability over less interpretable, marginally improved alternatives. Model performance across development and validation cohorts. Metric Cox model XGBoost MBoost RSF OST SuperLearner C-index (Development) 0.767 0.806 0.792 0.765 0.755 0.781 C-index (Mean, Validation) 0.745 0.726 0.727 0.667 0.693 0.746 5y Brier Score (Development) 0.478 0.478 0.475 0.523 0.491 0.475 5y Brier Score (Mean, Validation) 0.476 0.434 0.560 0.476 0.464 0.467 10y Brier Score (Development) 0.503 0.497 0.489 0.561 0.519 0.500 10y Brier Score (Mean, Validation) 0.480 0.450 0.510 0.457 0.470 0.470