Toward a Secure Future: An Integrated Framework for PII Detection Using NLP and Visual Analysis

Safeguarding Personally Identifiable Information (PII) is critical to prevent data breaches and identity theft in the era of digital services. This paper presents a hybrid PII detection and secure verification system that integrates rule-based regular expressions, transformer-based spaCy NLP, BERT models, and a Swin Transformer-based FasterRCNN pipeline. While text-based methods (Regex, SpaCy, BERT) excel at structured and contextual PII detection from OCR-processed text, the Swin Transformer provides vision-based detection capability for document images, handling complex layouts and visual cues often missed by traditional pipelines. It is observed that the empirical evaluation on a synthetic Indian PII dataset shows a significant performance improvement: the Regex method achieves an F1-score of 0.445, spacy reaches 0.166, and BERT alone achieves 0.042. The hybrid model, which combines these, yields an improved F1-score of 0.515, while the Swin Transformer model outperforms all others with an F1-score of 0.978, demonstrating near-perfect performance for image-based PII detection. A blockchain-backed double hashing mechanism, SHA-256, provides tamper-proof storage of PII fingerprints, while OTPbased authentication secures system access. In this work, it is proposed to use both OCR and vision-based detection in the scanned PDFs and images for robust and secure PII verification. This framework provides an interpretable, scalable, and privacy-preserving identity verification solution that improves upon the accuracy and layout awareness compared to the previous methods.