Efficient Embedding-based Synthetic Data Generation for Complex Reasoning Tasks

Read full text Summarise with VIRAT
Authors: Srideepika Jayaraman, Achille Fokoue, Dhaval Patel, Jayant Kalagnanam
Cited by: 0
References: 25

Summary

No summary yet. VIRAT can read this paper and write a plain-language summary of its aim, methods, key findings and limitations.

Summarise with VIRAT Free to try with your Science Catalogs account.

Abstract

Synthetic Data Generation (SDG), leveraging Large Language Models (LLMs), has recently been recognized and broadly adopted as an effective approach to improve the performance of smaller but more resource and compute efficient LLMs through fine-tuning. A key challenge in SDG is ensuring the quality and diversity of the generated data. In this paper, we analyze the diversity and distribution of generated data in the embedding space, and demonstrate a strong correlation between the density of examples within a specific neighborhood and the accuracy of predictions on examples drawn from that region. Building on this insight, we present a targeted pipeline for embedding-based sampling that enhances data diversity and consistently improves performance across several benchmarks.

Fields of Study & Subjects

Topic Modeling Domain Adaptation and Few-Shot Learning Natural Language Processing Techniques

Publication details

Journal / source
arXiv (Cornell University)
Publication date
15 March 2026
Language
EN
Document type
Preprint