Vision-Language-Action Model-Based Event-Triggered Admittance Control of a Mobile Manipulator for Power Substation Live-Maintaining

In this paper, for manipulating flexible objects, e.g., connecting a grounding wire with the power line, in live-maintaining of power substations, we propose an action-level vision-language model (VLM) for flexible object positioning, including a fuzzy-based dynamic motion primitive (DMP), an adaptive admittance model for reshaping the task manipulation trajectory, and a fuzzy logic system-based (FLS) adaptive controller with event-triggered mechanism (ETM) for dynamic uncertainties. First, a pretrained VLM is co-finetuned with vision data to construct an action-level VLM called vision-language-action model (VLAM). Such an end-to-end model can directly generate proper target poses on deformable linear objects (DLOs), without the need for complex segmentation modules. Second, an admittance control with stiffness and damping adaptation is proposed to enable the robot to effectively interact with unmodeled DLOs. Then, a learning from demonstration (LfD) strategy based on DMP with fuzzy clustering is designed to transfer the live-maintaining skill. Additionally, an FLS-based adaptive controller with ETM is proposed to compensate for complex nonlinear dynamics and reduce communication frequency. Finally, a mobile manipulator for power substation live-maintaining is exploited to validate the proposed approaches through extensive experiments.