Should Top-Down Clustering Affect Boundaries in Unsupervised Word Discovery?

We investigate the problem of segmenting unlabeled speech into word-like units and clustering these to create a lexicon. Prior work can be categorized into two frameworks. Bottom-up methods first determine boundaries and then cluster the fixed segmented words into a lexicon. In contrast, top-down methods incorporate higher-level information to inform boundary selection. Here, we explore the behavior of two similar approaches that differ in whether top-down information informs boundary selection. Our simple bottom-up strategy predicts word boundaries using the dissimilarity between adjacent self-supervised features, then clusters the resulting segments to construct a lexicon. Our top-down system is an updated version of the embedded segmental$K$-means (ES-KMeans) dynamic programming method that iteratively uses$K$-means to update its boundaries. On the five-language ZeroSpeech benchmarks, both approaches achieve comparable state-of-the-art results, with the bottom-up system being nearly five times faster. Through detailed analyses, we show that the top-down iterative re-segmentation in ES-KMeans can be beneficial (depending on factors like the candidate boundaries), but in many cases the simple bottom-up method performs just as well. For both methods, we show that the main limitations for boundary selection and lexicon learning come from the self-supervised representations used. Project code repository:https://github.com/s-malan/prom-seg-clus.