From kinetic theory to AI: A rediscovery of high-dimensional divergences and their properties
Abstract
Selecting an appropriate divergence measure is a critical aspect of machine learning, as it directly impacts model performance. Among the most widely used, we find the Kullback–Leibler (KL) divergence, originally introduced in kinetic theory as a measure of relative entropy between probability distributions. Just as in machine learning, the ability to quantify the proximity of probability distributions plays a central role in kinetic theory. In this paper, we present a comparative review of divergence measures rooted in kinetic theory, highlighting their theoretical foundations and exploring their potential applications in machine learning and artificial intelligence.
Fields of Study & Subjects
Statistical Mechanics and Entropy
Explainable Artificial Intelligence (XAI)
Ethics and Social Impacts of AI
Publication details
- Journal / source
- Mathematical Models and Methods in Applied Sciences
- Publisher
- World Scientific Pub Co Pte Ltd
- Volume
- 36
- Issue
- 06
- Pages
- 1185–1233
- Publication date
- 11 February 2026
- Language
- EN