Clarity Is Not Assumed: Understanding LLM-based Code Generation under Ambiguous Requirements

Read full text Summarise with VIRAT
Authors: Anonymous Author(s)
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

To support the empirical study of requirement ambiguity, we provide an integrated toolkit including the ORCHID benchmark datasets and accompanying evaluation pipelines. The toolkit contains function-level tasks with controlled injection of four ambiguity types (lexical, syntactic, semantic, and vagueness), together with modules for LLM-based code generation and analysis. It enables systematic evaluation of (1) performance impact under ambiguous requirements, (2) functional consistency across generations, and (3) ambiguity recognition and resolution capability. In addition, it provides utilities for quantifying performance degradation, functional divergence, and ambiguity recognition behavior across models.

Fields of Study & Subjects

Software Engineering Research Software Engineering Techniques and Practices Software Testing and Debugging Techniques

Publication details

Journal / source
Figshare
Publication date
25 March 2026
Language
EN