Chaotic and complex dynamics expose the limits of counterfactual reasoning

Abstract

Abstract Counterfactual reasoning, a cornerstone of human cognition and decision-making, is often seen as the “holy grail” of causal learning, with applications ranging from interpreting machine learning models to promoting algorithmic fairness. While counterfactual reasoning has been extensively studied in contexts with clearly defined static causal models, many real-world scenarios reside in dynamic settings often involving model and parameter uncertainty, observational noise, and chaotic behavior. The reliability of counterfactual analysis in such settings remains largely unexplored. In this work, we investigate the limitations of counterfactual reasoning in dynamic settings. We specifically focus on counterfactual sequence estimation and demonstrate empirically that even modest levels of model uncertainty or observational noise can lead to dramatic deviations between predicted and true counterfactual trajectories. Our findings urge caution when applying counterfactual reasoning in dynamical systems, particularly those that may exhibit complex, chaotic behavior, and highlight fundamental limitations in answering certain counterfactual queries reliably.