Adapter-RL: Adaptation of Any Agent Using Reinforcement Learning

This study introduces Adapter-RL, a novel architecture aimed at improving the performance of existing agents in reinforcement learning tasks. The approach integrates human-knowledge-based systems with deep reinforcement learning, combining the interpretability and rule-based logic of the former with the adaptive learning capabilities of the latter. A crucial aspect of this method is the use of “adapters”—concise modules integrated with a base-agent, designed to adjust the policy for specific tasks. The Adapter-RL framework comprises a base-agent responsible for initial decision-making and an adapter module that refines these decisions to meet task-specific requirements. The adapter facilitates efficient training, reduces parameter requirements, and mitigates catastrophic forgetting, enhancing overall performance and adaptability. This architecture enables agents to be fine-tuned effectively, allowing them to adapt to complex tasks with rapidly changing or uncertain conditions. The research demonstrates the efficacy of Adapter-RL through experiments in microRTS, a challenging real-time strategy game. The results demonstrate that Adapter-RL significantly accelerates the training process and outperforms base-agents across various tasks, highlighting its efficiency and robustness. In addition, the study investigates the temperature coefficient tradeoff in adapter training, finding that optimal performance is achievable within a broad range of coefficients. This underscores the stability of the method. The Adapter-RL method enables the specialization of base AI for specific characters or scenarios.