papers used for survey of automated GUI Testing
Abstract
we curated a diverse collection of datasets and benchmarks related to GUI functional testing in the context of LLM-powered agents. The evaluation corpus spans three primary categories: (1) public benchmarks that ensure scientific rigor and comparability, such as NCF Bugs Dataset, Android App Dataset, WebArena, AutoGUI-704k, ScreenSpot, VisualWebBench, Rico, and Defects4J; (2) expert-designed and our benchmarks (e.g., Custom ATA Benchmark, AutoPT Vulhub, Commit Benchmark, FestiVal, MAdroid, LLMDroid), which emphasize business relevance and specialized testing needs; and (3) real-world platforms and applications (e.g., to-do list managers, financial calculators, bookstores, and widely used Android apps) that assess robustness and adaptability beyond laboratory conditionsTogether, these datasets construct a multi-dimensional evaluation ecosystem covering bug reproduction, task and functional completion, and UI grounding and perception. This comprehensive dataset foundation not only enables a systematic mapping of current research but also establishes a reproducible and standardized platform for benchmarking future work in LLM-based GUI testing.
Fields of Study & Subjects
Software Testing and Debugging Techniques
Software Engineering Research
Robotic Process Automation Applications
Publication details
- Journal / source
- Figshare
- Publication date
- 1 January 2026
- Language
- EN