Measuring Conceptual Invariance: Four Experiments on the Creator-Independence Gradient

Read full text Summarise with VIRAT
Authors: Shreya Bhattacharya
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Abstract Large language models exhibit a domain-dependent performance gradient that is monotonic, replicable, and architecture-independent: near-perfect fidelity in formal domains and systematic degradation in domains characterised by authorial divergence. No existing evaluation framework predicts this gradient from a single measurable variable. This paper introduces Conceptual Invariance (CI), defined as the probability that a directed propositional connection is preserved under replacement of the authoring agent, and presents four experiments measuring it across textbooks, political philosophy, real AI conversations, and cross-model comparisons. Experiment 1 extracts 48 claims from four NCERT Class 11 textbooks spanning mathematics, physics, political science, and English literature. The measured CI gradient (1.0, 1.0, 0.25, 0.0) correlates with theoretically predicted values at r = 0.991 (p = 0.009, n = 4; illustrative of gradient direction rather than inferential given two extreme anchors). Inter-rater reliability on a 24-claim subset yields kappa = 0.92. Experiment 2 compares the propositional content of five political theorists (Hobbes, Marx, Mill, Rousseau, NCERT) responding to eight identical questions, using primary texts from Project Gutenberg. Mean cross-author agreement is 0.069. A physics control condition with five authors on an equivalent question yields agreement of 1.000. Experiment 3 analyses 253 Claude conversations from a single user (793,159 words, 225 days) across 11 domains, yielding CI values from 0.773 (code/technical) to 0.159 (personal reflection) with r = 0.997 (p < 0.001). The single-user corpus constrains generalisability of domain distributions but not of the CI gradient itself, which is a property of domains, not users. Experiment 4 validates the gradient across five models from four companies (Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.3, Gemini 3, Grok 4): 80 prompts across four domains yield cross-model similarity tracking the CI gradient at r = 0.955 (p = 0.045), with perfect Spearman rank ordering (rho = 1.0) in the four-model configuration using Claude Opus plus three cross-company models, which controls for shared training lineage within the Anthropic family. Two prior measurement attempts using TF-IDF and sentence embeddings produced inverted results (r = -0.95 and r = 0.894 respectively), establishing that CI is a propositional-semantic property that cannot be captured by lexical or distributional-semantic similarity measures. The convergence of four independent experimental paradigms on the same monotonic gradient constitutes evidence that the performance differential observed across AI systems is determined by a measurable structural property of the knowledge domain. Section 9.3 proposes the interference mechanism that explains why.

Fields of Study & Subjects

Topic Modeling Computational and Text Analysis Methods Natural Language Processing Techniques

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
19 March 2026
Language
EN
Document type
Preprint