LDDER: Learning Distinguishable and Dynamically Enhanced Representation for Visual Reinforcement Learning

Visual Reinforcement Learning (VRL) has demonstrated remarkable performance in vision-based control tasks by enabling agents to learn from high-dimensional observations. A critical challenge in VRL lies in extracting task-relevant information from visual observations into representations in the latent space. To tackle this challenge, VRL methods often employ the bisimulation metric, which measures differences in rewards and transitions between states. Despite its effectiveness, this approach suffers from representation collapse, where perceptually distinct states are mapped to similar embeddings, and fails to capture fine-grained temporal variations. In this paper, we propose Learning Distinguishable and Dynamically Enhanced Representation (LDDER), a novel VRL framework that explicitly enhances representation distinguishability and dynamic sensitivity. LDDER introduces a simple yet effective similarity minimization constraint applied at two levels: (1) a global constraint between arbitrary states to preserve perceptual diversity, and (2) a local constraint between adjacent states to enhance temporal sensitivity. Furthermore, to enhance dynamic modeling capacity, we integrate an inverse dynamics model into the framework, which enables the agent to better capture actioninduced transitions between states by predicting the actions. Crucially, the prior enforcement of distinguishability among states helps to capture clearer transition patterns, thereby reducing the difficulty of action prediction. Experimental results under three benchmark settings from DM Control Suite and CARLA indicate that LDDER consistently matches or outperforms state-of-the-art metric-based approaches across all evaluated tasks. Our code is available athttps://github.com/yangyyj/LDDER.