GUI-ARP: Enhancing Grounding with Adaptive Region Perception for GUI Agents

Read full text
Authors: Xuehan Ye, Yiqing Li, Wei Lie Dai, Miancan Liu, Ziyuan Chen, Zhangye Han, Hongbo Min, J. Ren, Xiantao Zhang, Yang Wen, Zhi Hao Jin
Cited by: 0
References: 30

Abstract

Existing GUI grounding methods often struggle with fine-grained localization in high-resolution screenshots. To address this, we propose GUI-ARP, a novel framework that enables adaptive multi-stage inference. Equipped with the proposed Adaptive Region Perception (ARP) and Adaptive Stage Controlling (ASC), GUI-ARP dynamically exploits visual attention for cropping task-relevant regions and adapts its inference strategy, performing a single-stage inference for simple cases and a multi-stage analysis for more complex scenarios. This is achieved through a two-phase training pipeline that integrates supervised fine-tuning with reinforcement fine-tuning based on Group Relative Policy Optimization (GRPO). Extensive experiments demonstrate that the proposed GUI-ARP achieves state-of-the-art performance on challenging GUI grounding benchmarks, with a 7B model reaching 60.8% accuracy on ScreenSpot-Pro and 30.9% on UI-Vision benchmark. Notably, GUI-ARP-7B demonstrates strong competitiveness against open-source 72B models (UITARS-72B at 38.1%) and proprietary models.

Fields of Study & Subjects

Context-Aware Activity Recognition Systems Robotics and Automated Systems Interactive and Immersive Displays

Publication details

Journal / source
ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher
IEEE
Pages
3676–3680
Publication date
21 April 2026
Language
EN