Interpretable Topic Modeling with Online Dirichlet Compound Negative Multinomial Mixture Model and Feature Saliency

The Dirichlet Compound Negative Multinomial (DCNM) distribution is a flexible probabilistic model used to model overdispersion and burstiness in multivariate count data. In this paper, we adopt a rising-factorial parametrization of the DCNM and develop a Minorization–Maximization (MM) algorithm for maximum likelihood estimation in a DCNM mixture model with feature saliency (DCNM-FS). The novel MM scheme yields closed-form updates for both mixture parameters and feature-saliency weights, improves numerical stability, reduces dependence on initialization, and avoids repeated evaluation of expensive derivatives and Hessians. To simultaneously select the number of components and the salient features, we use a convex combination of the Minimum Message Length (MML) and the Bayesian Information Criterion (BIC). The weighted criterion yields a more reliable model-selection signal for DCNM-FS mixtures by exploiting MML’s information-theoretic foundation while grounding it with BIC’s stronger sparsity-inducing tendency, resulting in more stable estimates of both the number of components and the salient feature set. Moreover, we introduce an online extension of DCNM-FS in which parameters and feature-saliency weights are updated incrementally from mini-batches, enabling streaming topic discovery without retraining from scratch. We embed the offline and online DCNM-FS mixtures in a modular topic-modeling pipeline, where the learned component parameters, in particular the feature saliencies, directly define interpretable topic keywords, reducing reliance on large language models for document representation. The proposed approach is validated on standard topic-detection benchmarks, where it demonstrates improved robustness, interpretability, and efficiency compared to existing clustering and topic-modeling baselines.