Bivariate Density Estimation: The Batched Broken Sample Problem and Stochastic Entropic Transfer Operators

This thesis focuses on bivariate density estimation in the context of systems of interacting particles. Given a joint probability distribution π∈𝒫(𝕏×𝕐) with marginals μ∈𝒫(𝕏) and ν∈𝒫(𝕐), we aim to estimate dπ/d(μ⊗ν), which isolates the dependence structure from the marginals, in contrast with the Lebesgue density of π. Two specific settings are considered. Discrete-time dynamical systems: The quantity dπ/d(μ⊗ν) can be interpreted as the transition density of a discrete-time stochastic dynamical system, which induces a stochastic Perron–Frobenius transfer operator. Given observations from a stationary deterministic system, a recent approach is to infer the entropic transfer operator from the samples, which is based on the regularising properties of entropic optimal transport plans. In this thesis, we generalise this method to stochastic and non-stationary systems and provide a quantitative convergence analysis of the empirical operator as the number of available samples increases. We introduce a method to extend the operator’s eigenfunctions to previously unseen samples, so that they can be efficiently incorporated into a spectral embedding. The practical utility and numerical scalability of the method are demonstrated on a real-world fluid-dynamics experiment. Broken-sample problem: Consider the broken random sample problem introduced by DeGroot, Feder, and Gole (1971, Ann. Math. Statist.). In each observation, a random sample of M point pairs {(Xᵢ,Yᵢ)}ᵢ (for i ∈{1,2,...,M}) is drawn from π. Suppose we observe only (Xᵢ)ᵢ and (Yᵢ)ᵢ separately; that is, the pairing information is lost. We are interested in estimating dπ/d(μ⊗ν) given N i.i.d. observations. Naïve maximum-likelihood estimation quickly becomes computationally intractable as M increases, owing to the combinatorial number of possible pairings. Instead, we propose a non-parametric pseudo-likelihood method based on entropic optimal transport and provide qualitative convergence results for the corresponding estimators. Moreover, in a parametric setting we prove a quantitative convergence rate for the maximum pseudo-likelihood estimator that is uniform in M. This enables estimation of dπ/d(μ⊗ν) even when the observed point clouds are very dense.