Rethinking Self-Supervised Learning in Computer Vision: Challenges, Tasks, and Methods

Self-supervised learning has become a cornerstone of modern computer vision and multimodal foundation models, enabling scalable training on massive unlabeled data and driving recent advances in perception for autonomous agents. However, despite impressive performance on high-level semantic recognition, these models remain brittle in dynamic, open-world physical environments that demand robust spatial and temporal reasoning. A key limitation is that spatiotemporal understanding is typically learned only indirectly from objectives targeting semantic recognition, rather than being treated as a foundational visual capability. In this talk, I will present our recent efforts to rethink self-supervised representation learning for vision including: (1) a comprehensive benchmarking of self-supervised learning methods across diverse non-semantic, mid-level vision tasks (2) a scalable framework that directly learn 3D geometry and scene dynamics from large-scale, in-the-wild video data (3) a method for learning 3D point cloud representations without any human-created 3D shapes