A Multimodal Generative AI Framework for Realistic Interview Simulation and Comprehensive Behavioral Analysis

Read full text Summarise with VIRAT
Authors: Eswar Raja, Ramya G R, Madhava Sudharsan S, Harish M
Cited by: 0
References: 7

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Abstract—The recruitment process is high-stakes, yet many candidates struggle due to anxiety and limited, unstructured feedback. Existing mock interview tools do not scale well and rarely capture behavioral signals beyond text. In this work, we present Interview Ai, a multi-modal system for interview simulation and performance analysis. The interaction is delivered through a conversational avatar that speaks and responds in real time, creating a more realistic setting than text-only systems. We model the interview flow as a finite state machine, which allows controlled transitions and adaptive follow-up questions. Candidate responses are evaluated using retrieval-based scoring, where answers are compared against exemplar knowledge rather than relying purely on model generation. The evaluation also incorporates measurable signals, such as filler word frequency, pauses, and response structure. In a study with 50 participants, users showed improvements in clarity, confidence indicators, and reduced hesitation compared to traditional practice. These results suggest that combining struc- tured control, grounded evaluation, and multi-modal interaction can provide effective and scalable interview coaching without replacing human decision-making. Index Terms—Large Language Models, LangGraph, Auto- mated Interview Systems, Computer Vision, Natural Language Processing, Speech Synthesis, GPT-4o-mini

Fields of Study & Subjects

AI and HR Technologies Speech and dialogue systems Intelligent Tutoring Systems and Adaptive Learning

Publication details

Journal / source
INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT
Volume
10
Issue
03
Pages
1–9
Publication date
30 March 2026
Language
EN