AI BASED EXAM MONITORING SYSTEM USING COMPUTER VISION

Purpose: Goal: Provide automated online exam monitoring using webcam + microphone to detect suspicious behavior and generate a post‑exam report. Main file: streamlit_dashboard.py How it works (High level): Camera frames captured via cv2 → analyzed by DeepFace for face detection/emotion and by ultralytics.YOLO (YOLOv8) for device detection → mediapipe face mesh used for gaze/iris landmarks → rule engine aggregates alerts → UI displays live metrics and writes alerts to final JSON repor nstallation / Requirements Install dependencies: Required key libraries: Streamlit: app UI and layout OpenCV (cv2): capture and drawing on frames DeepFace: face detection + emotion analysis ultralytics (YOLOv8): device detection using yolov8n.pt mediapipe: face mesh (gaze/iris landmarks) numpy, pandas: numeric handling and dataframes plotly: charts sounddevice (optional) and winsound (Windows) for audio checks/alerts Features (what we detect & how): Multiple faces: uses DeepFace.analyze() results, enhanced by size/location/aspect/overlap/distance filters plus persistence across frames to reduce false positives. Immediate exam termination after confirmed multi-face detection. Electronic devices (phones/tablets): YOLOv8 model (yolov8n.pt) detects devices. First detection = warning, repeated detection → terminate. Tab switching: JS visibility API posts messages to Streamlit; switching tab triggers immediate termination. Looking away / gaze: mediapipe face mesh estimates gaze. Previously shown as a metric; now optional/removed from UI but still tracked internally. Long continuous looking away (> GAZE_AWAY_THRESHOLD seconds) is considered a critical violation. Audio loudness: short RMS checks via sounddevice detect loud audio; adds multiple warnings. Head movement & emotion spikes: compared against baseline captured during calibration. These are logged for reporting but not always counted as warnings (configurable). Alerting & Termination Policy (current behavior): Configurable thresholds at top of streamlit_dashboard.py: MAX_WARNINGS — how many warnings before auto-end (default 15 after tuning) GAZE_AWAY_THRESHOLD — seconds looking away before alert (default 3) Only CRITICAL alerts count toward the warning limit by default: Counted: repeated electronic device, confirmed multiple faces, prolonged looking away, tab switch. Logged (not counted): minor emotion changes, small head movements, brief eye movements. Alerts are appended to session_data['alerts'] and included in the downloadable JSON report. Accuracy notes & limitations Face detection/emotion: accuracy depends on lighting, camera angle, occlusions; DeepFace performs well for frontal faces but may false-detect parts/reflections—mitigated by added validation (size, overlap, persistence). Device detection (YOLO): relies on model confidence; small/occluded phones may be missed, and false positives are possible with similar objects. Confidence threshold and NMS tuning help. Gaze detection (MediaPipe): reasonably accurate for coarse directions (left/center/right); fine-grained gaze (reading a specific line) is not reliable. Accuracy improves with higher landmark confidence and stable camera. Audio detection: RMS thresholding detects loud sounds but cannot attribute the source reliably (background vs student). Use only as a signal, not definitive proof. Overall system aims for conservative false-positive reduction (favor not ending exams wrongly) — implemented: multi-frame confirmation, overlap filtering, stricter thresholds. What's New / Recent Changes (delta): Reduced false positives for multiple-face detection: added face-size, edge filtering, overlap/distance checks and required persistence across several frames (5 consecutive frames by default) before alerting. Switched to counting only critical alerts toward termination and raised MAX_WARNINGS to prevent normal behavior from ending exams. Improved gaze/eye tracking: checking every frame, resetting saccade counts on a moving window, and capturing final looking-away time when an exam ends. Removed the live "Looking Away" metric from UI per request (still tracked internally) What's New / Recent Changes (delta): Reduced false positives for multiple-face detection: added face-size, edge filtering, overlap/distance checks and required persistence across several frames (5 consecutive frames by default) before alerting. Switched to counting only critical alerts toward termination and raised MAX_WARNINGS to prevent normal behavior from ending exams. Improved gaze/eye tracking: checking every frame, resetting saccade counts on a moving window, and capturing final looking-away time when an exam ends. Removed the live "Looking Away" metric from UI per request (still tracked internally) Where to find key code paths monitor_exam() in streamlit_dashboard.py: main loop for capture and alert handling. calibrate() in streamlit_dashboard.py: baseline capture for head/emotion. detect_multiple_faces() in streamlit_dashboard.py: improved multi-face detection logic. detect_electronic_devices() in streamlit_dashboard.py: YOLO device detector. detect_eye_gaze() in streamlit_dashboard.py: mediapipe face mesh gaze estimator. YOLO weights: yolov8n.pt in repo root. Data & Reports Live alerts stored in session_data['alerts'] and included in final JSON report downloaded by users. Example reports: files named like exam_report_pooja_YYYYMMDD_HHMMSS.json. Operational tips & tuning suggestions Lighting: ensure frontal, even lighting for best face/gaze accuracy. Camera: place at eye level; avoid strong backlight. YOLO tuning: increase conf threshold in detect_electronic_devices() to reduce false positives (e.g., 0.4–0.6). Face filter tuning: adjust multiple_faces_threshold (frames) and overlap percentage if you see missed detections or residual false positives. For production, consider persisting raw frames or short clip around alerts (privacy/consent permitting) for manual review. If you want stricter enforcement, move some behavioral alerts (e.g., repeated head movement) into the counted‑warnings list; otherwise keep them logged only.