A Coordinated Optimization Framework for Intelligent Agents With Online Evolutive Learning

As a prevalent field of study in machine learning, intelligent agents can perceive surroundings and make informed decisions. In many research areas such as autopilot systems, undersea explorations, and distributed robotics, researchers have traditionally employed unidirectional systems, which usually rely on perceptions from sensors to controllers, or end-to-end models, which generate actions directly from raw data. Nonetheless, in unidirectional systems, controller efficiency is intrinsically linked to sensor accuracy, which makes unidirectional systems lack a self-improving capability. Meanwhile, compared with functionally separated frameworks, end-to-end methods may have their own limitations in scalability, generality, interoperability, training costs, and so on. To fulfill this gap, we propose a new Coordinated Optimization Framework for Intelligent Agents (COIA). We introduce an inverted optimization channel from controllers to sensors in traditional functionally separated frameworks through communication between devices, enabling closed-loop online evolutive learning. To the best of our knowledge, this paper first presents a universal coordinated optimization framework among supervised learning and RL models, without human labels or intervention. Our method allows heterogeneous agents to autonomously adapt to some special situations in open environments, which forms a basis of networked Artificial General Intelligence (AGI). We design an experimental paradigm of COIA with concrete cases, which shows a significantly large performance margin over unidirectional and end-to-end models. The performance margin grows with task complexity.