Integrating AI and Large Language Models for Automated Data Quality Enhancement in Data Integration Systems

This paper introduces an AI and LLM-based framework to automate data quality improvement in complex data systems. Traditional methods struggle with semantic inconsistencies and evolving schemas, degrading quality as data scales. The framework incorporates Real-Time Semantic Annotation (RTSA), adaptive ontology reinforcement, contextual similarity for duplicate detection, and continuous auto-healing. Explainability is ensured via SHAP-based alignment for transparency. Evaluated on the GOBY Benchmark dataset, it achieved 89.4% semantic annotation accuracy, outperforming the strongest baseline by 3%. The duplicate reduction rate was 64.5%, and the quality score averaged 83.2%, validating the auto-healing loop's effectiveness. It adapts to evolving data without retraining, confirmed by robust performance under semantic drift. The explainability analysis showed a low SHAP divergence of 0.11, aligning model predictions with feature importance. An ablation study confirmed that the semantic feedback loop and ontology reinforcement significantly contribute to stability. This research offers a scalable solution for enhancing data quality by integrating AI-driven adaptability and LLM-based semantic understanding with built-in explainability for enterprise-grade data pipelines.