XAI-Based Document Verification System for Government Services and Job Applications

This paper proposes an XAI-based document verification system to automate the authentication of identity documents for government services and job applications, addressing the need for efficient and transparent verification in the digital era. The system integrates optical character recognition (OCR) tools, including EasyOCR and Tesseract, with machine learning models—text-based logistic regression using TF-IDF and image-based random forest classifiers—to process PDF uploads, extract text and images from documents like Aadhar Cards and PAN Cards, identify types, and verify authenticity. Explainability is enabled via Local Interpretable Model-agnostic Explanations (LIME), offering insights into model decisions to enhance trust. Trained on datasets of valid and fake documents, the system achieves 90% accuracy in identification and 80% confidence in detecting fakes, validated by a dual-model approach that tackles uncontrolled environment challenges, such as poor image quality, through preprocessing (grayscale, adaptive thresholding, rotations). Utilizing Python libraries like easyocr, pdf2image, and scikit-learn, it supports multilingual processing (English, Hindi) and concurrent verification, reducing processing time. It is tailored for public sector applications, such as caste certificate issuance and Peon/Office Attendant recruitment, aligning with global XAI trends in forensic and employment support systems. Future enhancements, including Structural Similarity Index (SSIM) for signature verification, are suggested to boost accuracy and scalability.