Directing gaze to maximize surface color information from natural scenes

Abstract

Abstract Natural scenes are often spatially and spectrally complex with a variety of land covers. Because of this complexity, where we look may not be guided by simple well-defined local image features or objects. Instead, gaze behavior may satisfy a more fundamental need: to reduce uncertainty about a scene’s elementary content, represented by the surface color at each point. The purpose of this study was to determine how effectively information is acquired by cone photoreceptors from scene fixations and how it relates to gaze behavior, scene variation, predictions of a deep neural network, and the number of available cone classes. Estimates of the information were computed from two sets of natural scene gaze data, one where observers searched for a target and the other where they viewed scenes freely. The information acquired in both tasks approached the maximum possible, with similar estimates from the network model. It increased with scene color diversity, quantified by color entropy, and decreased with fewer cone classes, as in some color vision deficiencies. These findings suggest that gaze is directed towards maximizing elementary scene information, which can then support more complex visual representations of scene content.