An Adaptive Hybrid Clustering Framework With Iterative Noise Filtering and a Novel DBNS Ratio Validity Measure for Outlier-Resilient High-Dimensional Data

Clustering high-dimensional data while assuring robustness to noise and outliers remains a significant challenge in unsupervised learning. This paper proposes an adaptive hybrid clustering framework that integrates the Mahalanobis distance in the latent space and iterative noise filtering to enhance outlier resilience. The framework employs an autoencoder to transform raw data into a compact latent representation, where clustering is performed using the Mahalanobis distance for outlier detection. An adaptive re-weighting mechanism iteratively minimizes the impact of noisy data points, refining cluster assignments. A hybrid loss function is proposed that integrates reconstruction accuracy, an outlier penalty, and the novel Davies-Bouldin Index Normalized Silhouette Score (DBNS) ratio. This combined loss optimizes clustering quality by jointly enhancing intra-cluster cohesion and inter-cluster separation. Experimental results on synthetic and real datasets demonstrate the superior robustness and precision of the proposed approach, outperforming the latest clustering methods in noisy environments.