UniDataBench: Evaluating Data Analytics Agents Across Structured and Unstructured Data

In real-world business environments, data is stored in a variety of sources, including structured relational databases, semi-structured databases, and unstructured files.The ability to extract reasonable insights across these diverse sources is integral to data-driven decisionmaking.Existing benchmarks, however, are limited in assessing agents' capabilities across these diverse data types.To address this gap, we introduce UniDataBench, a multi-source benchmark designed to evaluate the performance of data analytics agents in handling diverse data sources.Specifically, UniDataBench is constructed based on real-life industry analysis reports, employing a pipeline to synthesize data that aligns with authentic analytical trends.It encompasses diverse datasets spanning relational databases, CSV files, and NoSQL stores to reflect real-world business settings, and provides a unified framework for evaluating how effectively agents can explore multiple data formats, extract insights, and generate meaningful summaries and recommendations.Based on UniDataBench, we propose a novel LLM-based agent named ReActInsight, an autonomous agent that performs end-to-end analysis over diverse data sources by automatically discovering cross-source linkages, decomposing goals, and generating robust, self-correcting code to extract actionable insights.Our benchmark and agent together provide a framework for facilitating the development of data analytics agents in real-world applications.