Inaccuracies, omissions, and bullshit: 2024 U.S. elections through the eyes of ChatGPT

This study evaluated the accuracy, quality, and implications of outputs produced by generative artificial intelligences (AI) related to politics and electoral processes during the 2024 United States elections. We extend social media platform governance and content moderation theories to the study of AI, illustrating how large language models (LLMs), and the companies that own these technologies, shape and influence political information environments. Notably, we argue that the shift from social media to generative AI represents a fundamental evolution in platform power: a move from curative governance (providing information by sorting human speech) to generative governance (providing information by generating speech). Through a novel mixed-methods audit approach to studying ChatGPT-4o mini outputs regarding politics and elections collected less than two weeks before election day, we find that ChatGPT largely failed to provide users with accurate, reliable, and trustworthy information during a crucial election period. Our analysis identifies and categorizes potential democratic harms embedded in ChatGPT’s responses – such as inaccuracies, hedging, bullshit, and omissions – that paint a picture of a technology ill-equipped for democratic elections. This study illustrates the need for new frameworks for accountability in generative AI, particularly as voluntary industry safeguards fail to protect democratic information environments.