The Human Cerebellum Encodes Temporally Sensitive Reinforcement Learning Signals

Abstract

In addition to supervised motor learning, the cerebellum also supports nonmotor forms of learning, including reinforcement learning (RL). Recent studies in animal models have identified core RL signals related to reward processing, reward prediction, and prediction errors in specific regions in the cerebellar cortex. However, the constraints on these signals remain poorly understood, particularly in humans. Here, we investigated cerebellar RL signals in a computationally driven fMRI study. Human participants performed an RL task without low-level sensorimotor contingencies ( N  = 32; N female = 24). We observed robust RL signals related to reward processing and reward prediction errors (RPEs) in cognitive regions of the cerebellum. These signals were not explained by oculomotor or physiological confounds. By manipulating the delay between choices and reward outcomes, we discovered that cerebellar RL signals are temporally sensitive: robust when feedback was delivered shortly following choices but undetectable at suprasecond feedback delays. Similar delay effects were not found in other areas implicated in reward processing, including the ventral striatum and hippocampus. Furthermore, RPE activity in the cerebellum was related to behavioral performance when feedback was delivered promptly, but not when it was delayed. Connectivity analyses revealed that during RL feedback, cognitive areas of the cerebellum coactivated with a network that included the medial and lateral prefrontal cortex and caudate nucleus. Together, these results highlight a temporally constrained contribution of the human cerebellum to a cognitive learning task.