Assessing-Small-Language-Models-SLMs-for-Code-Generation: An Empirical Study with Benchmarks

### Research Overview This repository contains the code and evaluation framework for benchmarking 20 Small Language Models (SLMs) on 5 code generation benchmarks. This work extends the BigCode Evaluation Harness with automated benchmarking capabilities, VRAM monitoring, and performance tracking. Key Contributions Automated Benchmarking: Custom benchmarking.py script for systematic evaluation of multiple models Comprehensive Evaluation: 20 SLMs evaluated across 5 benchmark suites Performance Monitoring: Real-time VRAM usage tracking and execution time measurement Reproducible Results: Complete configuration and results for all experiments