Bayesian spatio-temporal modeling for policy evaluation: Sensitivity of policy effect estimates in the context of COVID-19 stay-at-home orders

Abstract

This study applies a Bayesian spatio-temporal model to demonstrate the sensitivity of policy effect estimates to spatial and temporal structure, using COVID-19 stay-at-home orders as a case study. Unlike conventional approaches, this framework accounts for geographic spillovers, temporal dependence, and space–time interaction, all of which are central to policy effect evaluation in heterogeneous settings. Implemented via Integrated Nested Laplace Approximation (INLA), the model also accommodates missing data and supports inference in high-dimensional contexts. Using Google mobility data and policy information from the Oxford COVID-19 Tracker, we estimate four models of increasing complexity: OLS, spatial, temporal, and spatio-temporal. While simpler models suggest substantial reductions in workplace and residential mobility, these effects become statistically insignificant once spatio-temporal interactions are incorporated. This pattern indicates that earlier studies may have overstated policy effects by overlooking spatio-temporal heterogeneity. Our findings demonstrate the importance of spatio-temporal modeling for policy evaluation, particularly when working with large-scale, incomplete, and unevenly distributed data.