Autonomous Causal Discovery: Evaluating LLMs’ Priors and Constraint Strategies for Reliability

Expert-guided Causal Structure Learning (CSL) incorporates prior knowledge to improve the accuracy of causal discovery, yet the acquisition of such knowledge is often restricted by the availability of human experts. While Large Language Models (LLMs) provide an alternative source of causal priors, LLM-derived knowledge can be inconsistent with the true causal structure due to hallucinations or contextual misinterpretations. This paper introduces a structural constraint measurement framework, which defines constraint strength and constraint quality to describe reliability and effectiveness, enabling a systematic evaluation of LLM-derived constraints. Using this framework, we evaluate five categories of structural constraints: Edge Existence (EEC), Edge Forbidden (EFC), Path Existence (PEC), Path Forbidden (PFC), and Order Constraints (OC). Our theoretical and empirical analyses demonstrate that while EEC offers high constraint strength, it exhibits low quality when derived from LLMs; conversely, PFC and OC provide a balanced trade-off between search-space pruning and reliability. Building on these insights, we propose a two-level CSL optimization framework that partitions the search space by node order and refines the structure using global path constraints. The results show that this framework provides an effective way to incorporate noisy LLM-derived priors into CSL, particularly in settings where expert knowledge is limited.