SplitWise regression for capturing nonlinear effects in interpretable model selection

Abstract

Abstract Capturing nonlinear relationships while maintaining interpretability remains a persistent challenge in regression modeling. We introduce SplitWise, a stepwise regression framework that adaptively transforms numeric predictors into threshold-based binary features using shallow decision trees—only when such transformations improve model fit according to the Akaike or Bayesian Information Criterion. This design preserves the transparency of linear models while flexibly capturing threshold-based nonlinear effects, positioning SplitWise between classical linear and interpretable nonlinear regression. SplitWise retains a single, globally linear equation that selectively incorporates data-driven thresholds—yielding models that remain straightforward to interpret and verify. Across synthetic scenarios with nonlinear signal patterns, SplitWise reduced median RMSE by 7–14% relative to the best-performing interpretable linear baseline and improved variable-selection accuracy (median Matthews Correlation Coefficient up to $$\sim$$ 0.79 vs.  $$\sim$$ 0.51 for LASSO). On real datasets, SplitWise matched or slightly improved RMSE while selecting fewer predictors. For instance, on Wine Quality (White), it improved RMSE from 0.756 to 0.752 and on Wine Quality (Red) from 0.654 to 0.649, using 6–10 predictors. On Bodyfat, it achieved 3.48–3.49 RMSE with four predictors, comparable to Elastic Net (3.41–3.48 RMSE) but with smaller models.