Self-Audit Module for Public Summarizers: PER, DSL, and Query Fidelity (v2) — Standing Metrics for Composition-Layer Self-Evaluation
Abstract
A standing metric specification and functional protocol for public summarizers (AI Overviews, chatbots, composition systems) to evaluate their own output using Provenance Erasure Rate (PER), Directionality of Semantic Labor (DSL), Query Fidelity Score (QFS), Erasure Skew (Omega-visible), and a combined Summarizer Audit Score (SAS). Designed to prevent self-flattery by anchoring every score to externally-derivable ground truth — the query string, cited sources, and named entity — rather than the summarizer's own assessment. Includes five calibration examples, eleven named failure flags, a keyword-preservation floor for framing detection, a GENERAL-knowledge quota, and hard-floor override rules.
Fields of Study & Subjects
Biomedical Text Mining and Ontologies
Topic Modeling
Scientific Computing and Data Management
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 3 June 2026