Software–Hardware Co-Design for Real-Time Speech Translation on Resource-Constrained NPUs
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
This paper presents a software–hardware co-design strategy for real-time streaming speech translation on resource-constrained NPUs. The proposed system is implemented on the alpha 11 AI Processor 4K Gen3 for 2026 OLED LG TVs, utilizing a single 1.8 TOPS NPU core out of the chip’s total 4.5 TOPS capacity. We optimize the deployment of large-scale Whisper and OPUS models through several key techniques: an asynchronous pipeline with Voice Activity Detection (VAD) and double-buffering; encoder-side KV projection relocation to eliminate redundant cross-attention computations; a hardware-aware mixed-precision quantization scheme; and memory access alignment tailored to the NPU microarchitecture. Validation on actual silicon demonstrates that our system processes a 5.12-second audio input in approximately 1.4 seconds, achieving a Real-Time Factor (RTF) of 0.276. Operating within a 620 MB memory footprint, this work provides a scalable solution for deploying high-performance foundation models on embedded System-on-a-Chip (SoC) environments without additional hardware costs.
Fields of Study & Subjects
Speech Recognition and Synthesis
Embedded Systems Design Techniques
Advanced Neural Network Applications
Publication details
- Journal / source
- IEEE Access
- Volume
- 14
- Pages
- 48901–48917
- Publication date
- 1 January 2026
- Language
- EN