CodeLSI: Leveraging Foundation Models for automated code generation with low-rank optimization and domain-specific instruction tuning

Background Automated code generation using Large Language Models (LLMs) offers promising solutions for enhancing software development efficiency. However, challenges remain in ensuring domain specificity, cost-effectiveness, and security—especially when relying on third-party APIs. This paper introduces CodeLSI, a framework that combines low-rank optimization and domain-specific instruction tuning to address these challenges. Objective The aim of this study is to develop and evaluate CodeLSI, a novel approach for generating high-quality code tailored to specific domains, using LLMs fine-tuned on company infrastructure without dependence on external APIs. Methods CodeLSI applies low-rank adaptation techniques to reduce the computational cost of model pre-training and fine-tuning. Domain-specific instruction tuning is employed to align code generation with organizational needs. We implemented and tested the framework on real-world JavaScript coding tasks using datasets drawn from internal software projects. Results Experimental evaluations show that CodeLSI produces high-quality, context-aware code. It outperforms baseline models in terms of relevance, accuracy, and domain fit. The use of low-rank optimization significantly reduced resource requirements, enabling scalable training on company-owned infrastructure. Conclusion CodeLSI demonstrates that combining low-rank optimization with domain-specific tuning can enhance the practicality and performance of LLMs for automated code generation. This approach provides a secure, cost-efficient alternative to commercial API-based solutions and supports faster, more targeted innovation in software development.