Hybrid ASR-T5 Architecture for Robust French Speech Recognition: Dual-Encoder Design for Production Systems

Read full text Summarise with VIRAT
Authors: Mohammad Alzahrani, Damien Forest
Cited by: 0
References: 2

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

We present a dual-encoder hybrid ASR-T5 architecture for French speech recognition that leverages both the encoder and decoder of pre-trained T5 language models -unlike existing approaches using only encoders (Whisper) or only decoders. A FastConformer acoustic encoder is combined with T5's full encoder-decoder via a lightweight adapter (~4M params), where the T5 encoder provides intermediate linguistic enrichment between acoustic encoding and generation. Controlled comparison on identical 10k-hour training data yields an 83.3% win rate over our production NeMo baseline across 18 evaluation categories. General-domain training achieves 3.9% validation WER with strong zero-shot generalization to the medical domain (22.78% WER), demonstrating architectural robustness rather than dataset overfitting.

Fields of Study & Subjects

Speech Recognition and Synthesis Speech and Audio Processing Speech and dialogue systems

Publication details

Journal / source
HAL
Publication date
14 March 2026
Language
EN
Document type
Preprint