Machine Learning under Scarcity: Addressing Data Scarcity in Tabular Models and Computational Scarcity in Statistical Estimation

Modeling tabular data remains a critical challenge, primarily due to fundamental issues of scarcity. This thesis presents a multi-faceted investigation into robust and efficient machine learning, unified by this central theme, tackling two primary forms: data scarcity and computational scarcity. To address data scarcity, two novel frameworks are developed. For tabular few-shot learning where all classes have severely limited training data (e.g., 5-shot or 1-shot), we introduce D2R2 (Diffusion-based Representation with Random Distance Matching). D2R2 pioneers a conditional diffusion model to learn robust semantic representations from unlabeled data. For imbalanced classification, where minority classes are vastly outnumbered by majority classes, we propose Invertible TabMap. This self-supervised framework learns an imbalance-robust latent space and utilizes an invertible network to achieve high-quality minority class synthesis. The thesis then addresses computational scarcity by tackling the fundamental inefficiency of nested simulation. Standard estimation methods in this domain are limited to a slow O(Γ−1/3) convergence rate. We establish a unifying theoretical framework: Least Squares Estimation (LSE) on a sieve. We then derive the precise conditions under which this framework achieves the optimal Monte Carlo convergence rate of O(Γ−1/2). Collectively, these contributions provide novel practical solutions for data scarcity in the tabular domain and advance the fundamental theoretical for overcoming computational scarcity in complex statistical estimation.