Replication Package for the submission "Hey Gemini, Could You Draw This Architecture? On Using LLMs to Generate Architectural Views" (ICSA'26-NEMI)

Read full text
Authors: Anonymised for double-blind revision
Cited by: 0

Abstract

Replication Package for the submission "Hey Gemini, Could You Draw This Architecture? On Using LLMs to Generate Architectural Views" (ICSA'26-NEMI)The package contains all artefacts necessary to reproduce the evaluation and analyses presented in this paper. Specifically, it includes:1. Evaluation Dataset (Excel File) An Excel file containing the full manual evaluation of all generated architectural views. The dataset includes:Repository identifiers and prompting strategy (e.g., 0-shot, 1-shot, few-shot)Descriptive assessment dimensions (Part I)Correctness evaluation scores across all ten semantic fidelity dimensions (Part II)Encoded penalty annotations used for qualitative error analysisThis file constitutes the ground-truth evaluation data used for all reported statistics and visualizations.2. Correctness Evaluation Framework A detailed document specifying the ten correctness evaluation dimensions, including their operational definitions. 3. View Generation Tooling (Python Project) A Python project used to generate architectural views for each prompting strategy. The project allows reproduction of the LLM-based generation step described in the paper. It includes its own README.md file with detailed setup instructions, dependency requirements, and execution guidelines. Together, these materials ensure transparency and enable full replication of the study’s analysis pipeline

Fields of Study & Subjects

Software Engineering Research Model-Driven Software Engineering Techniques Advanced Software Engineering Methodologies

Publication details

Journal / source
Figshare
Publication date
1 January 2026
Language
EN