From Single-Tool selection to Tool Pipelines: Extension and Evaluation of the Visual AI pilot VisionAgent
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
This thesis was developed in collaboration with Data Reply s.p.a. to explore the architecture and operational behaviour of the Visual AI Pilot Vision-Agent, a multimodal agent system designed by LandingAI to solve computer vision tasks through automated planning, code generation, and execution.Although agent-based systems of this kind offer a high degree of flexibility, their practical deployment often exposes limitations that are less visible at a purely conceptual level.In particular, issues related to reliability, behavioural predictability, and robustness tend to emerge when the system is used in interactive settings or when tasks require multi-step vision pipelines.The work begins with a detailed analysis of the original architecture and proceeds with the implementation of an application-level demo, including a frontend interface, a backend orchestration layer, and a controlled runtime environment.Importantly, this phase preserves the agent's original decision-making logic, allowing the system to be evaluated in a realistic usage context.Through empirical observation during this implementation phase, several architectural weaknesses become evident, such as ambiguous planning outputs, fragile tool selection mechanisms, inefficient handling of non-vision requests, and instability when composing complex pipelines.To overcome these limitations, the thesis proposes a set of targeted architectural extensions.These include a refactored Planner capable of producing structured and formally constrained pipelines, the introduction of a typed tool catalog to support controlled tool selection, explicit handling of non-vision cases to avoid unnecessary execution, and a redesigned Coder agent aligned with the structured planning outputs.The impact of these modifications is assessed through a controlled benchmark comparing the baseline and extended configurations.The experimental results indicate that the extended architecture achieves higher stability, improved correctness, and a significantly increased task success rate, particularly in multistep scenarios.At the same time, it reduces unnecessary executions and overall latency.Overall, the findings suggest that introducing structured planning and tighter execution control mechanisms can substantially improve the reliability and scalability of multimodal agent systems in real-world computer vision applications.
Fields of Study & Subjects
AI-based Problem Solving and Planning
Artificial Intelligence Applications
Multi-Agent Systems and Negotiation
Publication details
- Journal / source
- Electronic Theses and Dissertations Repository (University of Pisa)
- Publication date
- 22 February 2026
- Language
- IT
- Document type
- Thesis