The many faces of multivariate information

Extracting higher-order structures from multivariate data has become an area of intensive study in complex systems science, as these multipartite interactions can reveal insights into fundamental features of complex systems like emergent phenomena. Information theory provides a natural language for exploring these interactions, as it elegantly formalizes the problem of comparing "wholes" and "parts" using joint, conditional, and marginal entropies. A large number of distinct statistics have been developed over the years, all aiming to capture different aspects of "higher-order" information sharing. Here, we show that these functions are special cases of a more general function, $Δ^{k}$ which is parameterized by a free parameter $k$. Generally, the $Δ^{k}$ function is arranged into a hierarchy of increasingly high-order synergies; for a given value of $k$, if $Δ^{k}(\mathbf{X})>0$, then $\mathbf{X}$ is dominated by interactions with order greater than $k$, while if $Δ^{k}(\mathbf{X})