Orientation fields predict human perception of 3D shape from shading

Abstract

How the brain recovers the three-dimensional structure of surfaces and objects from 2D retinal images remains mysterious. Shading patterns provide one of the most powerful—yet least understood—visual depth cues. Most theories assume the brain infers surface normals from luminance values. However, this seems unlikely as visual neurons are broadly insensitive to luminance. To identify alternative cues, we measured responses of model orientation-selective cell populations to images of shaded objects. We found a surprising statistical relationship between image orientations and surface curvature properties, suggesting a way to estimate shape from shading. We find that the orientation-based cues not only predict striking illusions of shape perception when lighting varies, but also the impressive robustness of shape perception when large image modifications are introduced to directly pit luminance and image orientation cues against one another. The findings resolve the longstanding question of which image measurements drive shape from shading perception.