The Art of Computational Persuasion: Evaluating The Impact of Ethos, Pathos, and Logos on Argument Effectiveness

Large language models (LLMs) have emerged as a powerful tool for producing persuasive content at scale. However, little research has considered the rhetorical strategies which make these arguments effective. This study aims to fill this gap by quantifying the persuasive impact of rhetorical appeals in LLM-generated arguments, and personalization is additionally examined to determine whether it enhances their impact. Using a survey instrument with 68 participants to investigate which rhetorical appeal, when used by an LLM, is most convincing in both personalized and generic arguments. Participants were persuaded by the LLM’s arguments in 7.3% of instances. For personalized arguments, the rhetoric used did not have a significant impact on this metric; however, for generic arguments, appeals to logic were typically most convincing. These findings raise questions about the potential implications in politics and marketing if such persuasive content were deployed at scale.