Integration and response optimization algorithm of real time English speech translation system in multimodal interactive environment

This article explores the construction and response optimization algorithm of an English real-time speech translation system in a multimodal interactive environment. Voice translation aims to convert the speech content of a language into text or speech in the target language in real-time. Compared to traditional cascaded translation systems, end-to-end methods have significant advantages in delay control, error accumulation suppression, and resource utilization, and have received widespread attention in recent years. To improve the precision and response speed of real-time English speech translation in the current multimodal environment, this paper proposes an end-to-end speech translation system that integrates cross modal information. The system uses Transformer as the encoder and combines connected temporal classification (CTC) and attention mechanism as a hybrid decoder, taking into account the automatic alignment ability of speech and text and the acquisition of contextual attention weights, thus constructing the Transformer CTC/Attention model. The results show that the Transformer CTC/Attention model we constructed performs excellently in translation precision and response efficiency, effectively extracting speech keywords and achieving high-quality real-time translation of English speech.