Graph-Based Learning: From Graph Neural Networks to Large Language Models

Graphs provide a powerful representation for modeling complex relational data across domains such as social networks, recommender systems, chemistry, and biology. Recent advances in graph representation learning, particularly Graph Neural Networks (GNNs), have achieved notable success but continue to face limitations in scalability, robustness under noisy or incomplete structures, and integration with heterogeneous information. Meanwhile, Large Language Models (LLMs) have transformed natural language processing through their remarkable reasoning and generative capabilities, yet they remain poorly aligned with the structured, relational properties that are intrinsic to graphs. This tension has motivated increasing interest in bridging graph learning with LLMs to combine relational inductive bias with semantic reasoning. This thesis addresses these challenges from both methodological and application perspectives. First, we introduce LSGAT, which mitigates oversmoothing in deep GAT architectures by dynamically adjusting attention across layers. Second, we propose LangGSL, a variational mutual learning framework that couples LLMs with graph structure learning models, enabling robust representation learning under noisy and incomplete graphs. Third, we develop HyperG-VAE, a Bayesian hypergraph generative model tailored for Gene Regulatory Network (GRN) inference from single-cell RNA-seq data, which jointly models cellular heterogeneity and gene modules. Extensive experiments demonstrate that these approaches not only advance the methodological foundations of graph learning but also deliver real-world impact. In particular, LangGSL and LSGAT further strengthen graph representation learning across diverse settings, while HyperG-VAE provides biologically meaningful insights into B cell development. Collectively, this work contributes to the advancement of graph studies by strengthening fundamental models, broadening their applicability across diverse domains, and paving the way toward the next generation of scalable and generalizable graph learning frameworks.