AI-driven data lineage verification using temporal analysis with graph-based anomaly detection: A Comparative approach of supervised and unsupervised learning
Abstract
we developed an unsupervised approach using Graph Attention Networks (GATs) for structural representation learning and an Isolation Forest for ‘no-label’ anomaly detection. The model surrogate for validation produced over 99.8% accurate reporting on replicated anomaly patterns in unseen test data for Olist, TPC-H, and MIMIC-III. In the second phase, we developed a supervised, multi-modal framework that integrates GNNs, LSTM-Attention networks, DTW for automatic labeling, and Contrastive Learning. To counter the integrated class imbalance of the anomaly detection class, this framework incorporates the Synthetic Minority Over-Sampling Technique (SMOTE) as a fundamental component of its training. Comparing both models on three datasets, the unsupervised model outperforms the supervised model due to its ability to dynamically adapt to data without requiring labels. The supervised model achieves a maximum AUC of 0.96, and the Unsupervised model achieves an AUC of 1.00, indicating better prediction efficiency. The multi-faceted comparison of performance, feature importance, and operational dashboards provides the user with valuable insights, thereby confirming the effectiveness of the first unsupervised model and the second supervised multi-modal model, while fully retaining the explainability, governance, and scalability of data lineage in a comparative pair.
Publication details
- Journal / source
- Zenodo (CERN European Organization for Nuclear Research)
- Publication date
- 6 January 2026
- Document type
- Dataset