zeroknowledgediscovery/nero: NERO: Nonparametric Entropy-Rate Oracle for Long-Form AI Text Detection

This release provides the reference implementation, datasets, and analysis workflows for NERO (Nonparametric Entropy-Rate Oracle), a model-agnostic, training-free framework for characterizing long-form text via entropy-rate estimation. NERO estimates the Shannon entropy rate of text mapped to a fixed 27-symbol alphabet (26 letters plus space) using a nonparametric, PFSA-based method. When applied to long-form prose, entropy-rate estimates produced by NERO are systematically lower for contemporary large language model (LLM) outputs than for human-authored text. This intrinsic complexity signal enables robust discrimination between human and AI-generated text without access to model internals, token probabilities, or supervised retraining. This archive accompanies the paper: Schmidt R, Chattopadhyay I. Complexity Signature of Generated Text (2026). The release includes: the compiled NERO binary implementing the entropy-rate estimator, pipelines and metadata for generating long-form AI text (api_data_collection/), curated human reference corpora and categorizations (human_text_categories/), analysis notebooks reproducing all figures and results reported in the paper, derived tables and plot-ready data used for evaluation and visualization, manuscript sources for transparency and archival completeness. NERO is intended for long-form text analysis (on the order of 10⁴–10⁵ characters or more) and provides a principled, model-independent statistic for AI text detection and for tracking changes in generative behavior across model generations. License: Non-commercial