Replication Package for the submission "Hey Gemini, Could You Draw This Architecture? On Using LLMs to Generate Architectural Views" (ICSA'26-NEMI)
Abstract
Replication Package for the submission "Hey Gemini, Could You Draw This Architecture? On Using LLMs to Generate Architectural Views" (ICSA'26-NEMI)The package contains all artefacts necessary to reproduce the evaluation and analyses presented in this paper. Specifically, it includes:1. Evaluation Dataset (Excel File) An Excel file containing the full manual evaluation of all generated architectural views. The dataset includes:Repository identifiers and prompting strategy (e.g., 0-shot, 1-shot, few-shot)Descriptive assessment dimensions (Part I)Correctness evaluation scores across all ten semantic fidelity dimensions (Part II)Encoded penalty annotations used for qualitative error analysisThis file constitutes the ground-truth evaluation data used for all reported statistics and visualizations.2. Correctness Evaluation Framework A detailed document specifying the ten correctness evaluation dimensions, including their operational definitions. 3. View Generation Tooling (Python Project) A Python project used to generate architectural views for each prompting strategy. The project allows reproduction of the LLM-based generation step described in the paper. It includes its own README.md file with detailed setup instructions, dependency requirements, and execution guidelines. Together, these materials ensure transparency and enable full replication of the study’s analysis pipeline
Fields of Study & Subjects
Software Engineering Research
Model-Driven Software Engineering Techniques
Advanced Software Engineering Methodologies
Publication details
- Journal / source
- Figshare
- Publication date
- 1 January 2026
- Language
- EN