The Density Imperative, Refined: A 2x2 Ablation Reveals Independent Sufficiency of Density and Structure in Vision-Language Fine-Tuning
Abstract
% We previously reported that fine-tuning data quality is signed: sparse ~50-token captions catastrophically degrade a vision-language base model, while dense 111-field structured Neural Extraction of Semantic Topology (NEST) metadata enhances it. We now report a 2\times 2 ablation extension that decomposes that finding mechanistically. The original two-cell comparison conflated metadata density (tokens per record) with metadata structure (whether the supervision signal carries an explicit analytical schema). This paper closes the design with two new arms: Group~C (dense but unstructured) and Group~D (sparse but structured). Both novel arms are evaluated on the same Llama 3.2 11B Vision-Instruct base, the same 9{,}081-record Alexandria Aeternum training set, the same LoRA-32 hyperparameter envelope, and the same 100-image held-out CogBench protocol used in the original work. The result is sharper than the original framing: density and structure are independently sufficient to recover cognitive depth from the catastrophic Group~A regime. Paired Wilcoxon contrasts at n = 100 on the primary CogBench metric show Group~C (dense, unstructured) \approx Group~B (dense, structured) at \Delta = +0.008, p = 0.20; Group~D (sparse, structured) \approx Group~B at \Delta = -0.009, p = 0.24. Either density or structure alone matches the original "dense+structured" regime on cognitive depth; neither is required when the other is present. On secondary metrics, Group~D additionally wins: Group~D vs Group~B on capability retention (CCR): \Delta = +0.072, p = 0.004; Group~D vs Base on output information density: \Delta = +0.297, p = 3.9 \times 10^{-18}. Sparse data made structurally rich produces a measurable cognitive gain over the unmodified base, preserves base capability better than the dense-structured regime, and dramatically increases output information density. We additionally acknowledge an out-of-distribution scope refinement reported in companion work : on a 953-prompt sequestered OOD reasoning benchmark, the original 11.1x density advantage between Groups~A and~B produces zero measurable benefit on verifier-grounded accuracy or 4-judge Borda. The original in-domain claim is not falsified; its extrapolation to out-of-distribution reasoning is. The operational implication is concrete: do not train on sparse unstructured data; structure-rich sparse data is the cost-efficient operational regime where the base model already contains the underlying knowledge. We hypothesize that Group~D operates as an elicitation regime—reorganizing existing pre-trained associations into reasoning-accessible form rather than teaching new ones—and identify the falsifiable test that would bound this hypothesis (Group~D on a domain the base model has not seen, future work). All adapters, evaluation outputs, raw JSON aggregates, and a reproducibility harness are released for community verification.
Fields of Study & Subjects
Multimodal Machine Learning Applications
Ferroelectric and Negative Capacitance Devices
Domain Adaptation and Few-Shot Learning
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 13 May 2026
- Document type
- Preprint