Approach for Generating and Executing Acceptance Tests Based on Computer Vision and GPT-4

In recent years, large-scale language models (LLMs) have emerged as innovative solutions in various areas of software development. The creation of automated acceptance tests traditionally requires specialized technical knowledge and the involvement of multiple roles, resulting in waiting times and a high coding effort. This study presents an integrated system based on GPT-4 and computer vision for the generation and execution of ‘acceptance tests’ from natural language instructions. The system has two AI agents: the first generates tests in Gherkin format from textual descriptions, and the second executes the tests, interacting with the browser using GPT-4 and computer vision. The results demonstrate that the system can generate and execute consistent, functional, and reliable tests, helping to accelerate and democratize the test automation process.