A Discrete Language of Protein Words for Functional Discovery and Design

Project Overview: This repository contains the official dataset, evolutionary matrices, and pre-trained model weights for ProtWord, a novel discrete protein language model. Unlike traditional continuous-space protein language models, ProtWord utilizes a VQ-VAE architecture to compress complex 3D structural and evolutionary information into a discrete latent space consisting of an 8,192-token dictionary ("protein dialects"). With only 150M parameters, ProtWord demonstrates state-of-the-art efficiency in zero-shot variant effect prediction (VEP), remote homology detection, and multi-species evolutionary semantic analysis, effectively outperforming established tools like Foldseek and ESM-2 in specific benchmarks. Furthermore, this repository includes in vivo validation data. Utilizing ProtWord's functional predictions, we identified and validated a novel target, C7orf57 (ADMAP1). Included are the raw and processed Computer-Assisted Sperm Analysis (CASA) tracking datasets from our CRISPR-Cas9 knockout mouse models, which demonstrate severe sperm motility impairment and axonemal defects. Repository Contents: 📦 data.zip (Experimental & Evaluation Data) Evolutionary frequency matrices across 54 species. Semantic evaluation datasets for zero-shot VEP and structural alignment. Raw and processed tracking data from in vivo CASA (Computer-Assisted Sperm Analysis) characterizing the C7orf57 knockout phenotype. Other data could be used to reproduce the figures. License: CC BY 4.0 (Creative Commons Attribution 4.0 International). 🧠 checkpoints.zip (Model Weights & Codebooks) Pre-trained weights for the ProtWord-150M backbone. Pre-trained VQ-VAE codebook tensors (8,192 discrete tokens). Pre-trained weights for the latent GPT on Uniref50 and fine-tuned weights on Cofilin/ADF datraset License: ProtWord Open RAIL-M License. ⚠️ Biosafety Notice: The use of these model weights is strictly governed by Attachment A of the included Open RAIL-M license, which explicitly prohibits the generation or optimization of biological materials for the creation of human pathogens, biological weapons, or any highly hazardous synthetic entities. How to Cite: If you use the ProtWord model, codebooks, or the associated in vivo CASA datasets, please cite our corresponding preprint