From individual decisions to team emergence: a survey on explainable cooperative multi-agent reinforcement learning

Abstract Multi-Agent Reinforcement Learning (MARL) holds significant promise for cooperative decision-making, yet its reliance on deep neural networks (DNNs) creates ''black-box'' characteristics that impede trustworthy deployment in high-stakes scenarios. This lack of transparency complicates tracing decision logic and raises concerns about safety and accountability. This survey systematically reviews Explainable MARL (XMARL) for cooperative settings, deconstructing the decision-making chain from individual agent policies to collective team behavior. To address the absence of a unified framework, we introduce a novel multi-level taxonomy encompassing microscopic individual behavior, interaction mechanisms, team strategy emergence, and system-level performance. We organize core explanatory questions and technical approaches within this structure, summarize the principles and limitations of representative methods, and critically discuss key challenges such as evaluation standards, causal reasoning integration, and deployment adaptability. Our goal is to provide both theoretical foundation and technical guidance for building transparent, trustworthy, and verifiably cooperative multi-agent systems (MASs).