The Compiler as Oracle: Reliable LLM-Based Architecture Modeling via a Typed Internal DSL

Read full text
Authors: Irakli Betchvaia
Cited by: 2

Abstract

Large language models (LLMs) are increasingly used to generate software architecture models from natural-language descriptions, typically targeting untyped external DSLs such as PlantUML or Mermaid. Because these formats defer all validation to render time, LLM outputs frequently contain hallucinated references, dangling associations, and semantically inconsistent structures that are silently rendered or fail with uninformative parse errors. We propose a Generate-Compile-Repair (GCR) framework in which the target language is a statically typed internal DSL -- kUML, embedded in Kotlin -- and the Kotlin compiler serves as a deterministic, zero-cost validation oracle. A full run over a 50-task benchmark comparing kUML, PlantUML, and Mermaid shows kUML + GCR achieves the highest semantic fidelity and lowest hallucination rate of the three.

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
6 July 2026