Multi-Agent Path Planning Using Proximal Policy Optimization With Experience Sharing

Coordinating multiple autonomous agents in complex and dynamic environments remains a challenge. Traditional methods seldom account for inter-agent dependencies. Modern multi-agent machine learning approaches can suffer from high variance, slow convergence, and low sample efficiency. These challenges highlight the need for optimized learning mechanisms that can benefit from cooperation during training. To explore this idea, we investigate a multi-agent Proximal Policy Optimization (PPO) framework and examine whether sharing experience among agents can improve route-finding efficiency. In our simulation, two agents must reach predefined goals while avoiding collisions and forbidden zones. We compare two training schemes: a non-shared setup, where each agent maintains an independent actor–critic policy; and a shared experience setup, where both agents contribute to a common replay buffer and update a single shared actor-critic. Performance is evaluated through actor loss, critic loss, and return curves. The shared experience approach demonstrates less learning time and higher sample efficiency. Both variants converge to the same optimal policy.