SciCUEval: A Comprehensive Dataset for Evaluating Scientific Context Understanding in Large Language Models

SciCUEval is a benchmark designed to evaluate context understanding capabilities of Large Language Models (LLMs) in scientific domains. It addresses the lack of specialized benchmarks for scientific domains by providing: 10 Diverse Datasets: Spanning biology, chemistry, physics, biomedicine, and materials science. Multiple Data Modalities: Structured tables, knowledge graphs, and unstructured text. Four Core Competencies: Relevant information identification, Information-absence detection, Multi-source information integration, and Context-aware inference Comprehensive Evaluation: Assessing state-of-the-art LLMs across various scientific tasks.