Pseudo2Real: Task Arithmetic for Pseudo-Label Correction in Automatic Speech Recognition

Robust ASR under domain shift is crucial because real-world systems encounter unseen accents and domains with limited labeled data.Although pseudo-labeling offers a practical workaround, it often introduces systematic, accent-specific errors that filtering fails to fix.We ask: How can we correct these recurring biases without target ground truth?We propose a simple parameter-space correction: in a source domain containing both real and pseudolabeled data, two ASR models are fine-tuned from the same initialization, one on groundtruth labels and the other on pseudo-labels, and their weight difference forms a correction vector that captures pseudo-label biases.When applied to a pseudo-labeled target model, this vector enhances recognition, achieving up to a 35% relative Word Error Rate (WER) reduction on AFRISPEECH-200 across ten African accents with the Whisper TINY model.