Users perceptions of chatbot bullshitting

Context Since its original formulation, the Turing Test has served as a benchmark for evaluating machine intelligence based on linguistic behavior (Turing, 1950). Rather than assessing internal reasoning or problem-solving capabilities, the test focuses on whether a machine can produce language indistinguishable from that of a human in a text-based conversation. This emphasis on linguistic fluency as an indicator for intelligence reflects a long-standing human tendency to infer cognitive abilities from conversational competence. In practice, people are sensitive to surface-level linguistic cues such as coherence, grammaticality, and contextual appropriateness, often using them to evaluate the mental states or intelligence of others (Alter & Oppenheimer, 2009). As such, systems like large language models (LLMs), which produce fluent and natural-sounding language, are often perceived as more intelligent, even in the absence of genuine understanding or reasoning (Moussawi, Koufaris, & Benbunan-Fich, 2020). This behavior aligns with findings in social psychology and cognitive science, where language fluency is known to activate the fluency heuristic, leading people to equate ease of processing with competence and truth (Alter & Oppenheimer, 2009). The Turing Test thus highlights a key aspect of human behavior: the strong influence of communicative performance on attributions of intelligence, which remains relevant, but also potentially problematic, in the evaluation of today’s conversational AI systems. According to previous study participants presented a variety of expectations when discussing their interactions with chatbots. These expectations extended beyond the functional capabilities of the chatbot to encompass emotional and experiential aspects. User satisfaction seemed less tied to the actual accuracy or performance of the chatbot and more to the perception of its intelligence and competence. This phenomenon might be described as a form of "self-deception" or cognitive bias, where perceived competence outweighs actual reliability. The linguistic fluency of large language models (LLMs) often leads users to perceive them as more competent (Wester, de Jong, Pohl, & van Berkel, 2024). Their ability to produce coherent, context-sensitive, and persuasive language fosters the impression of cognitive depth and intentionality. However, this surface-level fluency masks a fundamental epistemic limitation: LLMs do not possess an understanding of the truth or a commitment to factual accuracy. Their responses are generated based on probabilistic associations in language, not on grounded knowledge or communicative intent. This phenomenon aligns with Harry Frankfurt’s (2005) Theory of Bullshit, in which he distinguishes bullshit from lying. Whereas liars deliberately distort known truths, bullshitters are indifferent to the truth and focus instead on producing statements that seem plausible or convincing. LLMs operate in a similar manner generating text that mimics coherent discourse without any regard for the veracity of the information presented. Their goal, implicitly, is not deception but fluency; they prioritize plausible language over factual correctness. This lack of epistemic commitment complicates human evaluations of LLM output, as users may mistake fluency for credibility, thereby overestimating the system’s intelligence and reliability. As such, LLMs embody a computational form of "bullshit": not by intention, but by design. Some psychological tendencies help explain why bullshit, as conceptualized by (Frankfurt, 2005), can be effective in social and human-computer interactions. The fluency heuristic, for instance, suggests that individuals judge statements as more truthful and reliable when they are articulated clearly and effortlessly (Alter & Oppenheimer, 2009). Language that is easy to process increases perceived credibility, even in the absence of evidence. This is compounded by authority bias, where people are more likely to accept information from sources that appear competent or confident, often equating fluency with expertise (Milgram, 1974; Kelley, 1973). Additionally, the Barnum effect, where individuals find vague, general, but positively framed statements personally meaningful, further enhances the persuasive impact of fluent but unsubstantiated language (Forer, 1949). When applied to interactions with large language models (LLMs), these biases help explain why users often interpret fluent, confidently presented responses as intelligent, accurate, and even personalized, despite the absence of actual understanding or epistemic commitment from the system. LLMs like ChatGPT, Bard, and other generative AI models do not have intrinsic knowledge. They predict the most statistically likely sequence of words based on their training data. This means that: • They do not "lie" intentionally (since they lack intent). • They do not verify truthfulness. They only aim for coherence and plausibility. • Their goal is to produce responses that sound right rather than be right. This aligns with Frankfurt’s definition of bullshit: LLMs are not concerned with truth but rather with the generation of plausible, fluent text. Language fluency refers to the perceived smoothness, coherence, and naturalness of linguistic output in human or artificial communication (McNamara, Crossley, & Mccarthy, 2010; Pitler & Nenkova, 2008). In the context of conversational agents, fluency encompasses grammatical accuracy, lexical appropriateness, discourse coherence, and prosodic or stylistic naturalness (Clark, Asli, & Smith, 2019). Large Language Model (LLM)-based chatbots, such as those powered by transformer architectures (e.g., GPT-4), typically generate highly fluent outputs due to their data-driven ability to model human-like language across diverse contexts (OpenAI, 2023; (Brown, Mann, Ryder, & Subbiah, 2020). In contrast, pre-scripted chatbots rely on manually written responses and rule-based branching structures, often resulting in more rigid, less adaptive interactions that can compromise perceived fluency (Bickmore & Sidner, 2006; Følstad, Nordheim, & Bjørkli, 2018). To systematically manipulate language fluency in an experimental setting, we can vary features such as syntactic complexity, lexical variety, and coherence between turns. For example, responses can be edited or selected to include or exclude disjointed discourse to create high- vs. low-fluency conditions (Krahmer & Theune, 2000). In addition to grammatical fluency and coherence, convincingness is an important characteristic of chatbot language that influences user trust, engagement, and compliance (Luger & Sellen, 2016; Fogg, 2003). Language generated by LLMs is often rated as more convincing than that of rule-based or pre-scripted systems, due to its ability to simulate human-like dialogue patterns, adapt tone dynamically, and maintain topical relevance (Song, Mamidisetty, Blanco, & Hong, 2025; Clark, Asli, & Smith, 2019). Several linguistic features contribute to this perceived persuasiveness: lexical richness, which conveys competence; contextual coherence, which enhances credibility and reduces perceived randomness; hedging and politeness strategies, which simulate social intelligence; and alignment with user intent, which makes the agent appear cooperative and knowledgeable (Narita & Kitamura, 2010). And convincing claims, basing in some claimed facts. Recent studies have shown that LLMs like GPT-4 can produce language rated as equally or more persuasive than human-written text in tasks like customer support, recommendation, or even negotiation (Kreps, McCain, & Brundage, 2023; Wei, et al., 2022). This effect is particularly strong when the system avoids overt errors, adapts its style to user expectations, and maintains a tone that is confident but not overassertive, traits that align with classical models of persuasive communication (Hovland & Weiss, 1951). Moreover, LLMs’ ability to generate personalized responses contributes to an impression of thoughtfulness, which enhances perceived intelligence and credibility (Wu, et al., 2025). To better understand these dynamics, we draw inspiration from the linguistic dimensions of social pragmatics proposed by (Deri, Rappaz, Aiello, & Quercia, 2018). Deri identifies ten dimensions that govern the way language conveys social relationships: Similarity, Identity, Knowledge, Social Support, Power, Respect, Romance, Trust, Fun, and Conflict. Translating these into the domain of LLM-mediated text-based interaction, we propose a framework that captures key linguistic cues used by LLMs to manage social meaning. We consolidate and adapt these dimensions into six pragmatic features commonly observed in LLM output: 1. Fancy words Using more elaborate or “fancier” words, such as saying alms instead of charity or donation, often leads others to perceive the speaker as more intelligent or educated (Oppenheimer, 2006). Example: As Gandalf might have said in The Lord of the Rings: “True wisdom is not in the gilded word, but in the light that makes meaning plain.” 2. Social Support / Fun (Emotive and Evaluative Language) The use of emotionally charged or evaluative language, such as adjectives ("wonderful," "devastating") and intensifiers ("deeply," "truly"), to foster a sense of support, empathy, or enthusiasm. Example: “Frodo’s resilience in Mordor is truly inspiring—it shows an extraordinary strength of character.” 3. Power (Appeal to Authority) Enhancing persuasive stance by referencing credible sources, general consensus, or expert opinion. This aligns with the social dimension of power by implying competence and authority. Example: “As Gandalf wisely noted, ‘All we have to decide is what to do with the time that is given us.’ This emphasizes the importance of agency.” 4. Similarity (Linguistic Alignment) Mirroring user’s lexical choices, tone, or