Reconsideration of Information-Theoretic Principles—Perspective from the Dual Probability Distribution
Abstract
We reconsider information-theoretic principles, such as the maximum entropy principle/minimum Massieu potential principle, from the perspective of the dual probability distribution. This is introduced through Sanov's Lemma for the multinomial distribution. The dual correspondence becomes asymptotically manifest. The Massieu potential is rewritten as the Kullback-Leibler divergence between the dual probability distribution and the dual reference distribution. Similarly, the dual potential is rewritten as the cumulant generating function with respect to the dual reference distribution. This perspective gives us new insight into information-theoretic principles. As the dual probability distribution naturally arises in data sampling, we anticipate that this new perspective will play a significant role in data analysis.
Fields of Study & Subjects
Statistical Mechanics and Entropy
Advanced Thermodynamics and Statistical Mechanics
Complex Systems and Time Series Analysis
Publication details
- Journal / source
- Entropy
- Publisher
- MDPI AG
- Volume
- 28
- Issue
- 6
- Pages
- 681–681
- Publication date
- 12 June 2026
- Language
- EN