Knowledge Package Quality as the Active Ingredient: Cross-Model Validation and Structural Comprehension Testing of a Knowledge System
Abstract
Testing whether a knowledge system's self-description actually transfers comprehension to a reader who has never seen it. We measured vocabulary transfer (97.7%), topology transfer (61.1% edge recall), and reasoning transfer (10/10 integration probes) using Gemini as an independent evaluator. We then tested whether improving knowledge package quality could close the topology gap — through structural augmentation (510-edge knowledge graph), targeted documentation fixes (section locality rule), and retrieval tool access (15 runs across 5 configurations including direct source code access). Key finding: a curated knowledge package outperforms both retrieval tools and direct source code access on a structural diagnostic comprehension test. A comparison against LLM-generated documentation produced a 23.7-point gap on a /50 scale (14.6 vs. 38.3), isolating curation quality as the active ingredient. Three empirically supported principles emerge: section locality determines retrievability, operational failure-mode content transfers better than structural description, and curation quality outperforms access breadth.
Fields of Study & Subjects
Information Retrieval and Search Behavior
Topic Modeling
Biomedical Text Mining and Ontologies
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 11 April 2026