Evaluating the quality of AI-generated subtitle translations from a reception-oriented perspective: a comparative study of ChatGPT, human, and neural machine translations in sitcoms

Research on audiovisual translation (AVT) has thrived in recent decades. However, the quality and reception of AI-generated subtitle translations remain relatively underexplored, despite the significant impact of artificial intelligence on traditional AVT methods. This study uses Pedersen’s FAR model, along with a viewer questionnaire survey, to evaluate the quality of subtitles generated by ChatGPT from a reception-oriented perspective. These subtitles are compared with those produced by human translators and neural machine translations for sitcoms. The results indicate that the clips subtitled by ChatGPT outperform those translated by Google Translate as evaluated by quality assessments and viewer responses. In some cases, the quality of ChatGPT-generated subtitles outperforms traditional neural machine translations and, in specific scenarios, can be comparable to or slightly outperform professional human translations, highlighting the potential of AI in subtitle translation. The educational background of viewers and their familiarity with the episodes significantly influenced the ratings of the subtitles. As a result, experts in translation are now exploring ChatGPT and its applications in translation. Additionally, postediting and thorough proofreading remain essential for ensuring the accuracy of AI-generated subtitle translations. This study demonstrates the feasibility of using artificial intelligence in sitcom subtitle translations and reveals critical factors from the audience’s perspective that deserve attention.