Visual Perception of 3D Shape: From Local 2D Image Measurements to 3D Surface Properties

Inferring 3D surface structure is one of the most fundamental functions of vision. There are many well-known depth cues, such as shading, texture, and highlights. However, how these cues are extracted from images-and what exactly they tell the brain about 3D shape-is not fully understood. Here, we describe how these seemingly distinct 3D shape cues could share a common currency for the first stages of shape estimation. The key insight is that when patterns such as shading or texture are projected from a 3D object into the 2D retinal image, they are spatially distorted, with profound consequences for local image statistics. The distortions create highly organized patterns of local image orientation (orientation fields) that are systematically related to specific 3D shape properties. Orientation fields can be reliably measured by filter populations and predict both successes and failures of human shape perception across diverse conditions.