InfiGUI-G1: Advancing GUI Grounding with Adaptive Exploration Policy Optimization

Read full text
Authors: Association for Artificial Intelligence 2026, Xiaotian Han, Xueyu Hu, Pengxiang Li, Zeyu Liu, Yuhang Liu, Jiasheng Wang, Xinyao Wang, Fei Wu, Congkai Xie, Hongxia Yang, Jianbo Yuan
+2 more Shengyu Zhang, Shuanghe Zhu
Cited by: 0

Abstract

The emergence of Multimodal Large Language Models (MLLMs) has propelled the development of autonomous agents that operate on Graphical User Interfaces (GUIs) using pure visual input. A fundamental challenge is robustly grounding natural language instructions. This requires a precise \textit{ spatial alignment}, which accurately locates the coordinates of each element, and, more critically, a correct \textit{ semantic alignment}, which matches the instructions to the functionally appropriate UI element. Although Reinforcement Learning with Verifiable Rewards (RLVR) has proven to be effective at improving \textit{spatial alignment} for these MLLMs, we find that inefficient exploration bottlenecks \textit{semantic alignment}, which prevent models from learning difficult semantic associations. To address this exploration problem, we present Adaptive Exploration Policy Optimization (AEPO), a new policy optimization framework. AEPO employs a multi-answer generation strategy to enforce broader exploration, which is then guided by a theoretically grounded Adaptive Exploration Reward (AER) function derived from first principles of efficiency $\eta=U/C$. Our AEPO-trained models, InfiGUI-G1-3B and InfiGUI-G1-7B, establish new state-of-the-art results across multiple challenging GUI grounding benchmarks, achieving significant relative improvements of up to 8.3\% against the naive RLVR baseline on benchmarks designed to test generalization and semantic understanding.

Fields of Study & Subjects

Multimodal Machine Learning Applications Reinforcement Learning in Robotics Speech and dialogue systems

Publication details

Journal / source
Underline Science Inc.
Publication date
7 January 2026
Language
EN