From Single-Tool selection to Tool Pipelines: Extension and Evaluation of the Visual AI pilot VisionAgent

Read full text Summarise with VIRAT
Authors: GUIDO TRENTACAPILLI
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This thesis was developed in collaboration with Data Reply s.p.a. to explore the architecture and operational behaviour of the Visual AI Pilot Vision-Agent, a multimodal agent system designed by LandingAI to solve computer vision tasks through automated planning, code generation, and execution.Although agent-based systems of this kind offer a high degree of flexibility, their practical deployment often exposes limitations that are less visible at a purely conceptual level.In particular, issues related to reliability, behavioural predictability, and robustness tend to emerge when the system is used in interactive settings or when tasks require multi-step vision pipelines.The work begins with a detailed analysis of the original architecture and proceeds with the implementation of an application-level demo, including a frontend interface, a backend orchestration layer, and a controlled runtime environment.Importantly, this phase preserves the agent's original decision-making logic, allowing the system to be evaluated in a realistic usage context.Through empirical observation during this implementation phase, several architectural weaknesses become evident, such as ambiguous planning outputs, fragile tool selection mechanisms, inefficient handling of non-vision requests, and instability when composing complex pipelines.To overcome these limitations, the thesis proposes a set of targeted architectural extensions.These include a refactored Planner capable of producing structured and formally constrained pipelines, the introduction of a typed tool catalog to support controlled tool selection, explicit handling of non-vision cases to avoid unnecessary execution, and a redesigned Coder agent aligned with the structured planning outputs.The impact of these modifications is assessed through a controlled benchmark comparing the baseline and extended configurations.The experimental results indicate that the extended architecture achieves higher stability, improved correctness, and a significantly increased task success rate, particularly in multistep scenarios.At the same time, it reduces unnecessary executions and overall latency.Overall, the findings suggest that introducing structured planning and tighter execution control mechanisms can substantially improve the reliability and scalability of multimodal agent systems in real-world computer vision applications.

Fields of Study & Subjects

AI-based Problem Solving and Planning Artificial Intelligence Applications Multi-Agent Systems and Negotiation

Publication details

Journal / source
Electronic Theses and Dissertations Repository (University of Pisa)
Publication date
22 February 2026
Language
IT
Document type
Thesis