Continuous Symbol Emergence via Decentralized Bayesian Inference With Gaussian Processes

Real-world communicative signals—such as gestures, vocalizations, and facial expressions—are inherently continuous and subtle. Research on emergent communication has been advanced as a constructive approach to elucidate the emergence of symbol systems enabling communication among agents through interactions mediated by physical signals, namely, signs. However, research on emergent communication using decentralized Bayesian inference has predominantly relied on discrete symbolic representations, struggling to capture inherent subtleties. To address this limitation, we propose a probabilistic framework called multimodal Gaussian process emergent communication (MGPEmCom), which enables the emergence of continuous symbols among decentralized agents. This framework integrates Gaussian process latent variable models to describe multimodal sensory observations and employs the Metropolis—Hastings naming game for decentralized inference, allowing agents to jointly infer shared continuous signs without accessing one another’s internal states. We validate our approach with a referential game task in which a speaker uses an emergent continuous symbol to communicate a target object to a listener. Experimental results demonstrate that MGPEmCom outperforms existing methods, including Inter-MDM and VAE+REINFORCE, particularly in scenarios that require fine-grained distinctions and generalization to unknown objects. These results indicate that our method enables more expressive and generalizable symbol emergence. This work establishes a principled Bayesian approach to decentralized symbol emergence using continuous latent representations, offering a foundation for modeling communication applicable to unknown objects.