Melody-Lyrics Matching With Contrastive Alignment Loss

The connection between music and lyrics is far beyond semantic bonds. Conceptual pairs in the two modalities, such as rhythm and rhyme, note duration and syllabic stress, and structure correspondence, raise a compelling yet seldom-explored direction in the field of music information retrieval. In this paper, we presentmelody-lyrics matching(MLM), a new task that retrieves potential lyrics for a given symbolic melody from text sources. Rather than generating lyrics from scratch, MLM essentially exploits the relationships between melody and lyrics. We propose a self-supervised representation learning framework with contrastive alignment loss for melody and lyrics. This has the potential to leverage the abundance of existing songs with paired melody and lyrics. No alignment annotations are required. Additionally, we introducesyllable vector, a novel representation for lyrics at the syllable-level activated by phoneme identity and vowel stress. We demonstrate that our method can retrieve lyrics for symbolic melody queries with empirical results and intuitive examples. We open-source code and provide matching examples on the companion webpage:https://github.com/changhongw/mlm.