Efficient Open-Vocabulary Visual Perception on Edge Devices via Multimodal Prompting
Abstract
YOLOE-Unified is a novel framework that integrates YOLOE with distilled CLIP, runtime SAM refinement, and TensorRT optimization for efficient open-vocabulary object detection and instance segmentation on edge devices (Jetson Orin, etc.). Highlights State-of-the-art zero-shot performance: 48.6% mAP (detection), 42.8% mask AP on LVIS Real-time on edge: 142 FPS (FP16), 118 FPS (INT8) on Jetson Orin NX Low power: ~14W average consumption Supports text, visual, and prompt-free modes
Fields of Study & Subjects
Advanced Neural Network Applications
Multimodal Machine Learning Applications
Advanced Image and Video Retrieval Techniques
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 9 January 2026
- Language
- EN