CO_EVO

Co-Evo: LLM-based Java Unit Test Co-evolution A research project studying how Large Language Models (LLMs) can assist in co-evolving Java unit tests when production code changes. Overview When production code evolves, corresponding unit tests often need to be updated to maintain test validity and coverage. This project investigates: RQ1: How do different context information levels affect test generation quality? RQ2: How do different ICL (Zero-shot, Few-shot, CoT, RAG) affect test generation quality? RQ3: How do different co-evolution types perform under optimal configuration? Project Structure src/co_evo/ ├── core/ # Shared components │ ├── executor.py # Test executor (Maven/JaCoCo integration) │ ├── extractor.py # Context extractor (Tree-sitter + LSP) │ ├── llm_utils.py # LLM utilities │ └── syntax_fixer.py # Java syntax auto-repair ├── rq1_context_information_effects/ # RQ1 experiments ├── rq2_prompt_strategies/ # RQ2 experiments ├── rq3_coevolution_types/ # RQ3 experiments ├── dataset_builder/ # 8-stage dataset construction pipeline ├── ground_truth/ # Baseline coverage collection ├── metrics/ # CodeBLEU evaluation ├── config/ # Configuration management └── utils/ # Utilities (logging, model, prompt, etc.) Dataset The dataset is constructed through an 8-stage pipeline: Stage Name Description Samples 1 Static Filter Extract candidates from Git history 15,545 2 Dynamic Verify Verify tests execute on both commits 8,973 3 Type Classify AST feature extraction & sampling 2,300 4 Ground Truth Collect coverage baseline 2,162 5 LLM Verify LLM classification verification 2,162 6 Confidence Filter Filter by confidence threshold 1,132 7 Human Review Expert manual review - 8 Export Final dataset export 530 Final Dataset Distribution By Co-evolution Type: Type Count Percentage Signature_Dominated 153 28.9% Pure_Logic 109 20.6% Dependency_Focused 95 17.9% Complex 94 17.7% Exception_Focused 79 14.9% By Repository: Repository Count Percentage Aiven-Open/klaw 107 20.2% alibaba/nacos 94 17.7% apache/shenyu 83 15.7% prebid/prebid-server-java 80 15.1% apache/rocketmq 65 12.3% shred/acme4j 51 9.6% OpenAPITools/openapi-generator 50 9.4% Installation Prerequisites Python 3.9+ Java JDK 8-22 (multiple versions for different projects) Maven 3.6+ Git Setup # Clone the repository git clone cd PythonProject # Create virtual environment python -m venv venv source venv/bin/activate # Linux/Mac # or: venv\Scripts\activate # Windows # Configure environment cp .env.example .env # Edit .env with your API keys and model configuration Environment Configuration Create a .env file with the following configuration: # Model selection MODEL_NAME=deepseek-v3 # API Keys (choose based on your model) DEEPSEEK_API_KEY=your_api_key_here # OPENAI_API_KEY=your_api_key_here # ANTHROPIC_API_KEY=your_api_key_here Usage All commands should be run with PYTHONPATH=src: RQ1: Context Information Effects # Run RQ1 experiments PYTHONPATH=src python -m src.co_evo.rq1_context_information_effects.main RQ2: ICL # Run RQ2 experiments PYTHONPATH=src python -m src.co_evo.rq2_prompt_strategies.main ### Ground Truth Collection ```bash PYTHONPATH=src python -m src.co_evo.ground_truth.main CodeBLEU Analysis # Initialize CodeBLEU environment (first time only) bash src/co_evo/metrics/codebleu/setup_env.sh # Run CodeBLEU analysis PYTHONPATH=src python -m src.co_evo.metrics.analyze_codebleu \ --dataset-file data/dataset_builder/stage8_final/final_dataset.json