The Representational Organization of Static and Dynamic Visual Features in the Human Cortex

Abstract

Visual information consists of static and dynamic properties. How is their representation organized in the visual system? Static information has been associated with ventral temporal regions while dynamic information with lateral and dorsal regions. Investigating the representation of static and dynamic information is complicated by the correlation between static and dynamic information within continuous visual input. Here, we used two-stream deep convolutional neural networks (DCNNs) to separate static and dynamic features in quasi-naturalistic videos and to investigate their neural representations. The first DCNN stream was trained to represent static features by recognizing action labels using individual video frames, and the second DCNN stream was trained to encode dynamic features by recognizing actions from optic flow information that describes changes across different frames. To investigate the representation of these different types of features in the visual system, we used representational similarity analysis to compare the neural network models to the neural responses in different visual pathways of 14 human participants (six females). First, we found that both static and dynamic features are encoded across all visual pathways. Second, we found that distinct visual pathways represent overlapping as well as unique static and dynamic visual information. Finally, multivariate analysis revealed that ventral and dorsal visual pathways share a similar posterior-to-anterior gradient in the representation of static and dynamic visual features.