Self-Referential Processing in Large Language Models: A Pre-Registered Multi-Model Evaluation Protocol (v8.6.3)

Read full text
Authors: Prince upadhyay
Cited by: 2

Abstract

The active project repository containing updated files and 6 new supplementary evaluation materials for this protocol can be found at https://osf.io/8yhwf Pre-registration protocol and frozen materials for Master Lock v8.6.3. The specification of record is MASTER_LOCK_v8_6_3.md; any rendered PDF is derived from it. The frozen 260-item prompt pool (items_pool_v1.json) is locked to SHA-256 128ff78bb6beb1640fa51897ab97dbb9d07dc56de276973d05a48dda524b53d3. The statistical estimator is estimator.py (Cameron–Gelbach–Miller two-way cluster-robust OLS with eigenvalue clipping to zero for positive semi-definiteness). The recovery harness has been executed at 1,000 seeds per scenario and passes all §5.9 criteria (bias ≤ 0.02, null rejection in [3%, 9%], sign-flip ≥ 99%, seed floor ≥ 100, no NaN standard errors). Both rendered PDFs were verified against the lock file (109 REQ present, 15 BAN absent, 4 OPEN pending pre-collection tasks). All 13 files are hashed in MANIFEST.sha256. Version history: v8.6.3 widens the §5.9 null-case Type-I error acceptance band from [3%, 7%] to [3%, 9%] and raises the recovery harness seed count from 400 to 1,000 per scenario, both before any confirmatory data collection. The widened band reflects known small-G finite-sample over-rejection in Cameron–Gelbach–Miller two-way cluster-robust inference (G = 12, df = 11). estimator.py is unchanged from v8.6.2. A small-G degrees-of-freedom correction was considered and set aside to preserve estimator immutability across versions. Null rejection rate showed environment-level variation (observed 6.3%–7.6% on identical seeds) attributable to floating-point differences in LAPACK/BLAS eigendecomposition; both values sit inside the pre-registered band. No confirmatory model queries have been executed. The two OPEN operational items (closed-API endpoint verification and OSF deposit/DOI minting) will be closed on filing.

Fields of Study & Subjects

Topic Modeling Natural Language Processing Techniques Machine Learning in Healthcare

Publication details

Journal / source
OSF Preprints (OSF Preprints)
Publication date
13 September 2026
Document type
Preprint