Pointwise Entropy Distributions for Community‐Level Hypothesis Testing in High‐Dimensional and Sparse Microbiome Data

ABSTRACT As techniques for linking bacterial genomes with the human microbiome have improved since the start of the Human Microbiome Project, various statistical and computational approaches in analyzing microbiome profiles have also developed over the past decades. One of the challenging tasks in this analysis is detecting a community‐level difference in microbiome data. Inherently, the count data of operational taxonomic units are sparse for most microbiomes, and classical tests for distribution differences under the normality assumption can hardly handle high‐dimensional microbiome profiles. We introduce a novel concept of pointwise entropy for the operational taxonomic units of microbiome, enabling permutation‐based hypothesis testing at the community levels. Through synthetic data and real‐world microbiome profiles related to human milk digestion designed for group comparison, we demonstrate its effectiveness in detecting community‐level differences. Our method offers a robust statistical and computational approach to the analysis of sparse, high‐dimensional microbiome data.