Reasoning Under Uncertainty in Multi-Agent Reinforcement Learning (MARL) with Upper Confidence Bound (UCB) Technique

This study explores the efficiency of the Upper Confidence Bound (UCB) algorithm in Multi-Agent Reinforcement Learning (MARL) to address the uncertainty in the exploration process. The primary focus is on how UCB facilitates efficient exploration by balancing exploration and exploitation, reducing uncertainty in the agent's decision-making process. We evaluate the algorithm's performance across single-agent, 2-agent, and 4-agent configurations, analyzing key metrics such as episode rewards, episode lengths, and learning stability. The results show that UCB effectively reduces uncertainty, enabling faster convergence and more efficient goal completion in both single-agent and multi-agent setups. While multi-agent configurations introduce additional variability due to agent interactions, UCB helps maintain stable learning and efficient exploration. Furthermore, the scalability of UCB is demonstrated, as the algorithm continues to function effectively even as the number of agents increases, ensuring that exploration remains efficient and manageable. Future work could explore larger agent populations, agent collaboration, and dynamic environments to further enhance the scalability and robustness of UCB in handling uncertainty.