Automating Latent Feature Auditing: Using Sparse Autoencoders to Identify and Neutralize Hallucination Traces in Small Language Models

Read full text Summarise with VIRAT
Authors: Harsh Raj
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Large Language Models (LLMs) often suffer from ”hallucinations”—generating factual errors with high confidence. This paper investigates the internal ”black box” of LLMs using Sparse Autoencoders (SAEs) to decompose activations into human-interpretable features. We propose a novel framework, Latent Feature Auditing (LFA), which identifies specific SAE features associated with factual incorrectness. By applying causal interventions, we demonstrate a reduction in hallucinations in the Gemma-2B model without requiring expensive retraining. Index Terms—Artificial Intelligence, Sparse Autoencoders, Mechanistic Interpretability, AI Safety, Hallucination Mitigation.

Fields of Study & Subjects

Explainable Artificial Intelligence (XAI) Adversarial Robustness in Machine Learning Topic Modeling

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
25 March 2026
Language
EN
Document type
Preprint