A Survey of RAG and Reasoning in Large Language Models

Retrieval-Augmented Generation (RAG) is an important development in the world of large language models (LLMs). By combining the ability of LLMs to generate text with access to external knowledge sources, RAG systems can answer questions and solve problems that require more than just what the model "remembers." As LLMs get better at reasoning, they allow RAG systems to handle more complicated tasks with higher accuracy and flexibility. In this survey, we carefully explain what we mean by "reasoning" in the context of RAG, and we provide a clear taxonomy that organizes different types of reasoning and retrieval strategies. We also show how reasoning and retrieval can work together to improve each other: better reasoning can guide more effective retrieval, and better retrieval can provide more relevant information for reasoning. Finally, we discuss new trends and promising directions for future research in this area. Overall, this work aims to give readers both a conceptual understanding and practical guidance for building more capable and intelligent retrieval-augmented systems.