Risk-Prioritized Experience Replay for Stable In-Hand Manipulation

Deep reinforcement learning (DRL) has shown strong capability for multi-finger dexterous in-hand manipulation, where high-dimensional control and complex object interactions make policy learning challenging. However, many existing DRL approaches emphasize task completion and learning efficiency without explicitly accounting for manipulation risk, which can lead to overly aggressive behaviors and unstable object handling. This study proposes Risk-Prioritized Experience Replay (Risk-PER), a replay-sampling strategy that incorporates task-specific risk scores derived from prior transitions. The proposed method assigns each transition a risk score based on three binary indicators related to manipulation instability and then biases replay toward lower-risk experiences while still allowing the agent to learn from risk-related events. Risk-PER is integrated with Deep Deterministic Policy Gradient (DDPG) and evaluated in MuJoCo simulation on two Allegro Hand in-hand manipulation tasks involving a block and an egg. Across the evaluated settings, Risk-PER achieves higher success rates, lower manipulation risk, and more stable learning behavior than HER and reward-penalty-based risk-averse baselines. These results suggest that incorporating task-specific risk awareness into replay prioritization can improve both learning efficiency and manipulation stability in dexterous in-hand manipulation.