Interplanetary Generalization and Adaptation of Deep Stereo on Mars

Stereo vision is critical for the autonomous operation ofthe Curiosity and Perseverance rovers on Mars. These rovers employa classical stereo algorithm based on block-matching to estimatedepth from a pair of navigation cameras, which is used to build a3D map of the terrain. The choice of this algorithm was motivatedprimarily by the computational constraints of radiation-hardenedflight hardware and heritage from previous Mars rover missions.However, recent data from Perseverance have revealed cases wherestereo perception was significantly degraded, leading to multipleinstances where autonomous navigation was no longer possible.This work analyzes the Flight Stereo performance of Perseveranceand identifies several contributing factors that lead to degradedperformance, primarily featureless terrain and imaging at low solarphase angle. Using a dataset curated from the navigation camerasover the entire Mars 2020 mission to date, we provide an empiricalanalysis of stereo performance under these conditions and evaluateadvanced learning-based methods that mitigate these failures.Representative deep stereo approaches, including cost-volume anditerative optimization methods, transformer-based architectures, andfoundation models, show strong zero-shot performance on real Marsimagery. We further present a domain adaptation strategy thatleverages foundation models as pseudo ground truth, showing itsclose alignment with Flight Stereo while providing dense disparityestimates. Finally, we analyze computational trade-offs to assessthe feasibility of deploying these algorithms on emerging spacequalifiedhardware. We hope these results will inform designconsiderations for stereo vision systems in future rover missions andcontribute toward more robust vision-based autonomous navigation.