Software–Hardware Co-Design for Real-Time Speech Translation on Resource-Constrained NPUs

Read full text Summarise with VIRAT
Authors: Geunjae Choi, Hongkyu Shin, Changjong Shin, Ellen Jieun Oh, Nojun Kwak
Cited by: 0
References: 22

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

This paper presents a software–hardware co-design strategy for real-time streaming speech translation on resource-constrained NPUs. The proposed system is implemented on the alpha 11 AI Processor 4K Gen3 for 2026 OLED LG TVs, utilizing a single 1.8 TOPS NPU core out of the chip’s total 4.5 TOPS capacity. We optimize the deployment of large-scale Whisper and OPUS models through several key techniques: an asynchronous pipeline with Voice Activity Detection (VAD) and double-buffering; encoder-side KV projection relocation to eliminate redundant cross-attention computations; a hardware-aware mixed-precision quantization scheme; and memory access alignment tailored to the NPU microarchitecture. Validation on actual silicon demonstrates that our system processes a 5.12-second audio input in approximately 1.4 seconds, achieving a Real-Time Factor (RTF) of 0.276. Operating within a 620 MB memory footprint, this work provides a scalable solution for deploying high-performance foundation models on embedded System-on-a-Chip (SoC) environments without additional hardware costs.

Fields of Study & Subjects

Speech Recognition and Synthesis Embedded Systems Design Techniques Advanced Neural Network Applications

Publication details

Journal / source
IEEE Access
Volume
14
Pages
48901–48917
Publication date
1 January 2026
Language
EN