Overlap Estimation to Hierarchical Clustering

In clustering, the measures used to quantify similarity between observations and clusters vary across methods. Density-based methods are well-suited for chained data, while partition-based methods perform better for elliptical-shaped clusters of relatively equal sizes. Identifying natural clusters in data with complex structures, such as weakly connected clusters of varying sizes, remains a significant challenge. A common heuristic in the literature assumes that measuring similarity between observations or groups is sufficient to reveal the overall structure of the data. However, this paper demonstrates that this assumption does not always hold true. We formally quantify multi-component overlap among clusters, inspired by human visual inspection, and propose a new clustering method based on this quantification. The proposed approach effectively addresses the challenge of clustering data with both chained structures and clusters of unequal sizes. Preliminary experiments demonstrate the method’s effectiveness in geometric clustering and color image segmentation tasks.