A single reinforcement learning model to unify habit formation and Pavlovian-instrumental interaction

Abstract

Abstract The conditioning paradigm has observed learning behaviors in animals. The theory which explains behaviors assumes that a brain appropriately combine two reinforcement learning systems, consisting of model-based and model-free system. There are two ways to combine systems: hybrid system, in which two systems work independently and are combined, and single system, in which two systems work cooperatively. While the former is widely discussed, the latter has not been sufficiently investigated in terms of its capabilities. Here, we propose a single system that integrates model-based and model-free systems to include the roles of Pavlovian conditioning. Our simulation results show that the proposed model can explain habit formation, Pavlovian conditioning, and Pavlovian-to-instrumental transfer in a unified manner. This suggests the possibility of reproducing various phenomena in a single system.