ClarifyCoder: Replication Package

Read full text Summarise with VIRAT
Authors: "Anonymous"
Cited by: 0

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Replication package for the paper "ClarifyCoder: Fine-Tuning Code LLMs for Clarification-Seeking in Ambiguous Code Generation Tasks." This package contains: - Training data (data/FINAL_finetuning_data_ques_only.jsonl): 29,896 synthetic (problem, clarifying-question) pairs across three ambiguity types (ambiguous, inconsistent, incomplete), generated using Gemini-1.5. Fine-tuning script (fine-tuning/clarify_aware_fine_tuning_v2.py): Clarify-aware instruction tuning of DeepSeek-Coder-6.7B-Instruct using LoRA (PEFT). Evaluation pipeline (human-eval-comm/): Scripts for running the two-round HumanEvalComm evaluation protocol, including question generation, GPT-4o-mini Q&A evaluation, code generation, and metric aggregation. Benchmark (human-eval-comm/Benchmark/HumanEvalComm_v2.jsonl): 771 evaluation instances across 7 ambiguity variants (1a, 1c, 1p, 2ac, 2ap, 2cp, 3acp). Experiment runner (run_humaneval_comm_experiments.sh): Reproduces all experimental conditions: DeepSeek Coder baseline, ClarifyCoder, Okanagan, and ClarifyCoder+Agent.

Fields of Study & Subjects

Software Engineering Research Topic Modeling Teaching and Learning Programming

Publication details

Journal / source
Zenodo (CERN European Organization for Nuclear Research)
Publication date
25 March 2026
Language
EN