MoFaaS LLM Feedback Loop

MoFaaS LLM Feedback Loop This repository holds the service that transforms the code from one version to a different version of the same FaaS function. The repository is organized into two branches: main: Contains the functional blocks of the LLM Feedback Loop. evaluation: Includes tests and results obtained during the evaluation phase. Components backend: Related to the LLM Agent in the paper. code-verifier: Related to the Syntax Agent. create-test-service: Related to the Building Agent. runtime-verifier: Related to the Functional Testing Agent. frontend: Related to the Entry Point. The system is designed as a feedback loop, with the backend (LLM Agent) commanding all other components. Therefore, the backend needs to be set up with the URLs of all other agents. Installation Backend For any execution, you will need to access the backend directory. At the moment, the image is using the LLM model phi-4-Q4_K_M. You must download it to the models directory: mkdir models cd models wget https://huggingface.co/bartowski/phi-4-GGUF/resolve/main/phi-4-Q4_K_M.gguf You will have to change the environment variables in the docker-compose.yaml according to your environment. You should set: CODE_VERIFIER_URL: the URL for the service code-verifier CREATE_TEST_URL: the URL for the service create-test-service CODE_TESTER_URL: the URL for the service runtime-verifier For the backend, you have two options: Run in a machine with NVidia and CUDA: You will need to install the CUDA Container Toolkit in the host. Then, simply run the Docker Compose with: docker compose -f docker-compose.yaml -f docker-compose-gpu.override.yaml up Run in a machine without GPU: Run the following in your terminal: docker compose -f docker-compose.yaml up Helm Chart Everything but the backend is packaged under an Helm chart. This Helm Chart needs to be launched in a Kubernetes cluster with Knative installed (refer to the MoFaaS main repository for instructions on how to install Knative): cd chart helm install version-generation . All containers are already provided in the repository. However, the backend repository might need to be built for the specific machine where it will be run, as the llm-cpp-python is a compiled library. Contributing We welcome contributions! Please open an issue or submit a pull request. Licensing This repository is licensed under the GNU GENERAL PUBLIC LICENSE 3.0. For the complete license text, see the file LICENSE. This license applies to all files in this distribution, except as noted below. The Open Chat Playground in the frontend directory is licensed under the MIT license. See the frontend/LICENSE.md for the complete license text.