4D-aware stereo matching via implicit spectral reconstruction with multi-modal training and RGB-only deployment
Abstract
Conventional RGB stereo matching suffers from ambiguities caused by metamerism and weak textures. We demonstrate that implicitly recovered spectral information can enhance depth estimation without requiring spectral sensors at deployment. A reconstruction-to-matching architecture maps RGB inputs to a latent spectral space, injecting material priors into stereo matching. To address data scarcity, we develop a co-aperture system combining a liquid crystal tunable filter with Scheimpflug LiDAR for pixel-aligned multimodal acquisition, and propose a "Measure-and-Complete" strategy using sparse LiDAR to generate dense pseudo-ground truth. Experiments show 4.34% reduction in endpoint error, validating the effectiveness of spectral priors for geometric perception.
Fields of Study & Subjects
Advanced Vision and Imaging
Robotics and Sensor-Based Localization
Advanced Image and Video Retrieval Techniques
Publication details
- Journal / source
- Optics Express
- Publisher
- Optica Publishing Group
- Volume
- 34
- Issue
- 13
- Pages
- 23284–23284
- Publication date
- 14 May 2026
- Language
- EN