Self-Audit Module for Public Summarizers: PER, DSL, and Query Fidelity (v2) — Standing Metrics for Composition-Layer Self-Evaluation

Read full text
Authors: Lee Sharks
Cited by: 16

Abstract

A standing metric specification and functional protocol for public summarizers (AI Overviews, chatbots, composition systems) to evaluate their own output using Provenance Erasure Rate (PER), Directionality of Semantic Labor (DSL), Query Fidelity Score (QFS), Erasure Skew (Omega-visible), and a combined Summarizer Audit Score (SAS). Designed to prevent self-flattery by anchoring every score to externally-derivable ground truth — the query string, cited sources, and named entity — rather than the summarizer's own assessment. Includes five calibration examples, eleven named failure flags, a keyword-preservation floor for framing detection, a GENERAL-knowledge quota, and hard-floor override rules.

Fields of Study & Subjects

Biomedical Text Mining and Ontologies Topic Modeling Scientific Computing and Data Management

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
3 June 2026