VibeBench: An Automated Framework for the Holistic Evaluation of LLM-Generated Code

Based on your current repository state and the specific version of VibeBench you have prepared, here is a professional title and description for your GitHub release. This is designed to satisfy JOSS reviewers by highlighting the framework's "feature-complete" status and its research-oriented goals. Release Title v1.0.0: Initial Research Release - Holistic LLM Evaluation Framework Description Summary This is the official stable release of VibeBench (v1.0.0), an automated benchmarking suite designed to evaluate Large Language Models (LLMs) beyond functional correctness. This version introduces integrated static analysis through Abstract Syntax Tree (AST) heuristics and sandboxed dynamic execution to measure code maintainability, security risks, and resource efficiency. Key Features Static Quality Heuristics: Implements the CodeAnalyzer to detect "AI-isms," including ghost comments, hardcoded credentials, and documentation gaps. Sandboxed Execution: Utilizes a secure CodeExecutor with Unix-based resource limiting (CPU/Memory) to benchmark runtime stability. Benchmark Reference Data: Includes a formalized Human Baseline (Gold Standard) in datasets/human_samples/ for comparative performance grounding. Automated Reporting: Features the VibeReporter for generating high-fidelity Markdown leaderboards and JSON research artifacts. Technical Updates JOSS Compliance: Includes a complete paper.md, LICENSE (MIT), CONTRIBUTING.md, and CODE_OF_CONDUCT.md. Testing Infrastructure: Added a suite of automated unit tests in tests/test_core.py. Dependency Management: Restructured requirements.txt to distinguish between core operational tools and research analysis extensions. How to Use Clone the repository. Install dependencies: pip install -r requirements.txt. Run the benchmark: python vibebench.py. Generate report: python -m core.reporter.py