Physical mechanisms governing generalization and hallucination in deep learning for imaging through scattering media

Abstract Deep learning has revolutionized computational imaging, yet its real-world deployment remains constrained by two critical challenges: poor generalization under dynamic conditions and the emergence of hallucinatory artifacts. By leveraging a physics-guided framework based on scattering media, a model system where controlled variations in light transmission matrices ( $$T$$ T ) isolates these challenges, we unravel the mechanistic interplay between generalization limits and hallucination origins. We demonstrate that a network’s generalization capacity is fundamentally bounded by its ability to accommodate distinct inverse mappings ( $${T}^{-1}$$ T − 1 ), while hallucinations arise when this capacity is exceeded, resulting in unconstrained, non-physical predictions. We also identify residual ballistic light, if not negligible, as a stabilizing anchor, enabling robust predictions under scattering variability. Integrating experimental validation with wave-optics simulations, we establish a universal framework that links these phenomena, showing that strategic training on diverse physical mappings enhances generalization while suppressing hallucinations. This work bridges physics-driven interpretability with AI design, offering actionable strategies to develop reliable models for applications ranging from medical imaging through biological tissues to autonomous navigation in scattering environments.