Knowledge Package Quality as the Active Ingredient: Cross-Model Validation and Structural Comprehension Testing of a Knowledge System

Read full text
Authors: John Wade
Cited by: 2

Abstract

Testing whether a knowledge system's self-description actually transfers comprehension to a reader who has never seen it. We measured vocabulary transfer (97.7%), topology transfer (61.1% edge recall), and reasoning transfer (10/10 integration probes) using Gemini as an independent evaluator. We then tested whether improving knowledge package quality could close the topology gap — through structural augmentation (510-edge knowledge graph), targeted documentation fixes (section locality rule), and retrieval tool access (15 runs across 5 configurations including direct source code access). Key finding: a curated knowledge package outperforms both retrieval tools and direct source code access on a structural diagnostic comprehension test. A comparison against LLM-generated documentation produced a 23.7-point gap on a /50 scale (14.6 vs. 38.3), isolating curation quality as the active ingredient. Three empirically supported principles emerge: section locality determines retrievability, operational failure-mode content transfers better than structural description, and curation quality outperforms access breadth.

Fields of Study & Subjects

Information Retrieval and Search Behavior Topic Modeling Biomedical Text Mining and Ontologies

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
11 April 2026