papers used for survey of automated GUI Testing

Read full text
Authors: Son Ngo
Cited by: 0

Abstract

we curated a diverse collection of datasets and benchmarks related to GUI functional testing in the context of LLM-powered agents. The evaluation corpus spans three primary categories: (1) public benchmarks that ensure scientific rigor and comparability, such as NCF Bugs Dataset, Android App Dataset, WebArena, AutoGUI-704k, ScreenSpot, VisualWebBench, Rico, and Defects4J; (2) expert-designed and our benchmarks (e.g., Custom ATA Benchmark, AutoPT Vulhub, Commit Benchmark, FestiVal, MAdroid, LLMDroid), which emphasize business relevance and specialized testing needs; and (3) real-world platforms and applications (e.g., to-do list managers, financial calculators, bookstores, and widely used Android apps) that assess robustness and adaptability beyond laboratory conditionsTogether, these datasets construct a multi-dimensional evaluation ecosystem covering bug reproduction, task and functional completion, and UI grounding and perception. This comprehensive dataset foundation not only enables a systematic mapping of current research but also establishes a reproducible and standardized platform for benchmarking future work in LLM-based GUI testing.

Fields of Study & Subjects

Software Testing and Debugging Techniques Software Engineering Research Robotic Process Automation Applications

Publication details

Journal / source
Figshare
Publication date
1 January 2026
Language
EN