CrossTemporal: Domain-Adaptive Temporal Video Alignment for Egocentric Task Guidance in Augmented Reality

Temporal video alignment-learning to find framelevel correspondences between videos of similar activities-has emerged as a powerful self-supervised pretext task for video representation learning. However, existing approaches assume that training and deployment videos share the same visual domain, limiting their applicability in real-world augmented reality (AR) task guidance, where reference demonstrations are typically recorded from a fixed third-person viewpoint while user videos are captured from head-mounted egocentric cameras. In this paper, we propose CrossTemporal, a domain-adaptive temporal video alignment framework that transfers temporally structured representations across visual domains. CrossTemporal extends the Soft-DTW and Contrastive-IDM alignment framework [8] with two novel components: (1) a Domain-Invariant Temporal Encoder (DITE) that learns view-invariant frame embeddings through adversarial domain alignment while preserving temporal structure, and (2) a Temporal Anchor Bridging (TAB) mechanism that establishes cross-domain temporal correspondences using a small set of automatically detected anchor frames. We evaluate CrossTemporal on three cross-domain alignment scenarios: thirdperson to egocentric, single-viewpoint to multi-viewpoint, and studio to in-the-wild. Experiments on Assembly101, H2O, EGTEA Gaze+, and the Pouring and IKEA ASM benchmarks used in [8] demonstrate that CrossTemporal reduces cross-domain alignment error by 23-38% compared to direct transfer baselines and enables accurate real-time step tracking in egocentric AR task guidance, extending the capabilities of systems such as RetroActivity [18]. Our architecture is compatible with the video frame correlation system described in [1], providing a pathway to production deployment.