Auditing Fairness in RAG-Augmented LLM Interview Systems: A Study of Ollama-Based Local Deployment for Automated Technical Hiring
Summary
No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.
Summarise with VIRAT Free to try with your Science Catalogs account.Abstract
The growing adoption of Large Language Models (LLMs) in automated technical hiring systems raises critical concerns regarding fairness, transparency, and evaluation stability. While Retrieval-Augmented Generation (RAG) enhances contextual grounding by incorporating resume-specific knowledge, bias auditing in locally deployed LLM infrastructures remains largely underexplored. This paper presents a fairness auditing framework implemented within a containerized architecture using Ollama running Llama3, orchestrated through Docker-based microservices. The system integrates a Next.js frontend, a Node.js backend for interview orchestration, PostgreSQL-based semantic resume retrieval, and Redis for stateful multi-turn memory management, forming a privacy-preserving RAG-enabled technical linter view platform. To systematically evaluate bias, we design controlled resume perturbation experiments that isolate demographic indicators, institution a background, and linguistic fluency while preserving equivalent technical skills. The system generates adaptive interview questions and structured performance scores across multiple simulated sessions. Fairness is quantified using score variance analysis, difficulty drift measurement, sentiment polarity comparison, and multi-run stability metrics. Statistical comparisons are performed to detect evaluation inconsistencies under controlled variable shifts. To mitigate observed disparities, prompt-neutralization strategies and rubric-based scoring calibration mechanisms are introduced and re-evaluated experimentally. Results demonstrate measurable bias patterns and corresponding reduction trends after mitigation, contributing a reproducible applied AI engineering framework for auditing and improving fairness in RAG-based, locally deployed LLM interview systems.
Fields of Study & Subjects
Ethics and Social Impacts of AI
AI and HR Technologies
Names
Identity
and Discrimination Research
Publication details
- Journal / source
- Research Digest on Engineering Management and Social Innovations
- Publication date
- 31 March 2026
- Language
- EN