Communication Learning in Multi-Agent Systems From Graph Modeling Perspective

In numerous artificial intelligence applications, the collaborative efforts of multiple intelligent agents are imperative for the successful attainment of target objectives. To enhance coordination among these agents, a distributed communication framework is often employed, wherein each agent must be capable of encoding information received from the environment and determining how to share it with other agents as required by the task at hand. However, indiscriminate information sharing among all agents can be resource-intensive, and the adoption of manually pre-defined communication architectures imposes constraints on inter-agent communication, thus limiting the potential for effective collaboration. Moreover, the communication framework often remains static during inference, which may result in sustained high resource consumption, as in most cases, only key decisions necessitate information sharing among agents. In this study, we propose a novel approach where the communication structure between agents is represented as a learnable graph.We frame this challenge as the task of identifying the optimal communication graph while allowing the architecture parameters to be updated through regular optimization, which requires a bi-level optimization process. By applying continuous relaxation to the graph structure and integrating attention mechanisms, our method, CommFormer, effectively optimizes the communication graph and simultaneously refines the architectural parameters via gradient descent in an end-to-end manner. Additionally, we introduce a temporal gating mechanism for each agent, enabling dynamic decisions on whether to receive shared information at a given time, based on current observations, thus improving decisionmaking efficiency. Comprehensive experiments conducted across a range of cooperative tasks demonstrate the robustness of our model. Our approach enables agents to develop more coordinated and sophisticated strategies, maintaining effectiveness even with varying agent counts.