Emergence of Biological Structural Discovery in General-Purpose Language Models

ABSTRACT Large language models (LLMs) are evolving into engines for scientific discovery, yet the assumption that biological understanding requires domain-specific pre-training remains largely unchallenged. Here we report that general-purpose LLMs possess an emergent capability for biological structural discovery. Under strict, shortcut-controlled evaluation, a small-scale GPT-2 (124M) fine-tuned solely on English paraphrase discrimination detects protein homology zero-shot at ROC-AUC 0.79 on a shortcut-controlled benchmark. Controls establish that the ability is conferred by pre-training , not architecture: a randomly initialized GPT-2 is at chance (0.52). To exclude the possibility that public checkpoints were contaminated with biological data, we train our own GPT-2 from scratch on an English-only web corpus; it reproduces the transfer (0.76), proving the effect arises from linguistic pre-training alone. Network-based interpretability reveals a deep structural isomorphism: the discriminative signal localizes to deep layers (0.97 at layer 9), and attention analysis surfaces modality-agnostic “difference” operators. Scaling to massive instruction-tuned models further improves performance, including in the remote-homology “twilight zone”, which we report as an exploratory upper bound because those models’ training corpora are undisclosed. We formalize these tasks through the BioPAWS benchmark. Our controlled results—obtained entirely on models with known training data—establish that abstract logical structures distilled from human language constitute a genuine, if bounded, cognitive prior for decoding the syntax of biology.