Synergy of giants and specialists: A large-and-small model integration framework driven by generative AI for smart manufacturing

The advent of Generative Artificial Intelligence (GenAI) foundation models, particularly Large Language and Vision Models (LMs), is poised to revolutionize smart manufacturing. However, their application is hindered by hallucinations and lack of domain-specific grounding. Conversely, traditional Small Models (SMs) are precise but struggle with unstructured inputs like engineering drawings. To bridge this gap, this paper proposes a synergistic Large-and-Small Model (LSM) integration framework specifically for automated machining process planning. In this framework, a "Giant" domain-adapted Vision-Language Model (VLM), Large Language and Vision Assistant (LLaVA) serves as a perception front-end, translating complex 2D engineering drawing and geometric tolerances into structured entities. These entities ground a "Specialist" reasoning system comprising a Transformer-based Knowledge Graph (KG) embedding model (TransEx) and a Deep Reinforcement Learning (DRL) agent guided by a Generalized Advantage Estimation-Proximal Policy Optimization (GAE-PPO) strategy. This synergy enables interpretable, multi-hop reasoning to autonomously determine optimal machining routes and cutting parameters. Experimental results demonstrate the framework's efficacy: the VLM achieves a Process Recognition Accuracy of 83.24%, and the complete pipeline enhances knowledge inference accuracy by 13.51% and cumulative reward by 49.01% compared to baseline methods. This work presents a robust framework where the synergy between "Giants" and "Specialists" offers a scalable path toward advanced industrial automation.