Information sampling and Bayesian belief formation in statistical judgment

Abstract

The statistical properties of data are often communicated using visual graphs, like scatterplots. However, decision makers make systematic errors when processing these graphs, with important consequences for statistical communication in science, medicine, and policy. We propose that decision makers are Bayesian learners, who learn optimally given the data points that they attend to. Accordingly, judgment errors arise from biased sampling of information from graphs. We tested our theory in four eye-tracking experiments (total N = 421), in which participants made correlation judgments from scatterplots of both experimentally manipulated data (Experiment 1) and real data (Experiment 2), as well as plots with different display formats (Experiments 3 and 4). Participants’ judgments displayed several known biases, including underestimation of absolute correlations and sensitivity to irrelevant visual features. Importantly, the (optimal) Bayesian belief updating model, trained on the sensory inputs from visual information search, predicted both participants’ judgments and associated biases with high accuracy in all the experiments. Additionally, a computational model of participants’ information sampling processes, combined with the Bayesian model, reproduced all behavioral regularities. These results shed light on the cognitive mechanisms of belief formation, show how statistical judgments can be quantitatively predicted and manipulated, and provide insights for data visualization and statistical communication.