Visual scene drawing through human–robot natural language interactions using lightweight large language models

Abstract Human–robot interaction increasingly leverages large language models (LLMs) to enable intuitive, text-based communication between humans and robotic systems. However, most existing LLM-driven robots rely on cloud computation or large-scale models that are difficult to deploy on embedded robotic platforms with limited resources. This work investigates a lightweight, fully on-device alternative in which a compact local LLM interprets natural-language scene descriptions and generates geometric instructions for a six-degree-of-freedom (6-DoF) robotic drawing arm. We propose a text-to-geometry pipeline that converts user-provided scene descriptions into structured two-dimensional geometric layouts composed of labeled primitives with metric dimensions and spatial anchors defined within a fixed drawing workspace. These geometric representations are expressed as explicit coordinate sets and subsequently transformed into robot joint trajectories through analytical inverse kinematics. The resulting trajectories are executed by a physical 6-DoF robotic arm, establishing a direct mapping from natural language to executable robot geometry without reliance on images, perception modules, or external computation. Evaluation results demonstrate that lightweight local LLMs can generate consistent geometric representations suitable for robotic execution across diverse scene descriptions. The robotic system successfully reproduces multi-object geometric scenes from language-derived coordinates, validating the feasibility of on-device natural-language-driven physical drawing. Observed limitations, including occasional geometric misinterpretation by the language model and robot actuation resolution constraints, highlight practical trade-offs and design considerations for deploying lightweight LLMs in real-world robotic platforms.