Mapping the Collaboration between Crowdsourcing and Large Language Models: A Fine-Grained Survey

With the widespread application of Large Language Models (LLMs) in Natural Language Processing (NLP), their collaboration with crowdsourcing is becoming increasingly close. Traditional crowdsourcing models primarily undertake foundational tasks such as data collection and annotation. However, with the rapid advancements in LLMs technology, their capabilities in handling complex tasks have significantly improved. LLMs can now feedback to crowdsourcing platforms by generating tasks, verifying results and providing feedback, reshaping the core workflows of the crowdsourcing ecosystem and enhancing task design and operational efficiency. At the same time, the unique strengths of crowdsourcing in harnessing human knowledge and judgment offer crucial support for LLMs training and optimization. This is especially evident in key processes such as large-scale training data construction, human preference alignment, and model performance evaluation. As a result, the relationship between LLMs and crowdsourcing has evolved from a one-way dependency into a dynamic, collaborative, and mutually beneficial paradigm of human–AI integration. In this context, this paper presents the first comprehensive survey that spans both the LLMs and crowdsourcing domains. It aims to systematically review and analyze how LLMs empower crowdsourcing processes in areas such as task execution and quality control, while also examining the essential role of crowdsourcing in model training, feedback alignment, and evaluation. Furthermore, it identifies key challenges in current research and applications, and explores future directions for advancing human–AI collaborative intelligence and developing governance mechanisms for crowdsourcing. The papers surveyed in this work are available at: https://github.com/Ikaros-sc/crowdsourcing.