A From-Scratch 774M Model That Says "I Do Not Know": Native Unknown Emission and Boundary-Preserving Knowledge Refresh (v1.2)

Read full text
Authors: Chao Qin
Cited by: 8

Abstract

Trained from scratch on ~4.7M programmatically generated tokens (no pretrained initialization), a 774M model emits 'unknown' as a first-class output class with held-out generalization 0.94 on novel concepts (baseline 0.65) after root-conflict adversarial training. A dual-channel fusion with a character-trigram entry index (entry-level AUC 0.98) achieves zero unknown-side leakage across 25 probes while recovering false rejections. A daily-refresh protocol (fresh entries + current-OOD adversarial batch + replay in ONE optimization pool) passes three release gates (new-knowledge >=85%, OOD interception >=7/8, legacy regression >=95%) in a single 14.5-minute incremental round on one consumer GPU. To the best of our knowledge this is the first published sub-billion from-scratch model with native unknown emission evaluated on held-out generalization.

Fields of Study & Subjects

Adversarial Robustness in Machine Learning Generative Adversarial Networks and Image Synthesis Advanced Neural Network Applications

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
31 August 2026
Language
EN
Document type
Preprint