Detecting Code from Evolving LLMs: Temporal Drift, Generalization, and Adaptation

Generative AI has found use in a variety of fields, including in software development. As a result, it has become important to develop tools to identify LLM-generated code. However, LLMs evolve rapidly, and it can be expensive to keep detectors up to date through frequent data collection and retraining. We evaluate generalization across LLMs, with particular attention to temporal differences between model generations. Our findings demonstrate a generalization gap, but that lightweight mitigation methods can reduce the gap at reduced cost compared to training new detectors from scratch. We highlight approaches for maintaining cost-efficient and interpretable LLM-detectors as new models emerge and capabilities evolve.