Security of Large Model-based Agents: A Survey on Adversarial, Poisoning, and Backdoor Attacks

The rapid advancement of artificial intelligence (AI), especially for large model (LM) based agents, powered by large language models (LLMs), large vision models (LVMs), and multimodal foundation models, has led to transformative progress in autonomous reasoning, decision-making, and human-AI interaction. However, the integration of large models within agentic AI frameworks also introduces novel security challenges. In this survey, we provide a comprehensive review of security threats in LM-based agents, with a particular focus on adversarial attacks, data poisoning, and backdoor attacks. We first explore the fundamentals of LM-based agents and present a taxonomy of security attacks In particular, we explain unique attack interfaces, datasets, and benchmarks within the agentic AI system. We then systematically review recent advances in attack methodologies and corresponding defense strategies across both individual large models and multi-agent settings. Moreover, we also summarize their common objectives and distinct characteristics, focusing on execution phases, required attacker knowledge, persistence, and potential impact on agentic AI systems. Finally, we discuss current challenges and emerging research trends, including robust cross-model and multi-agent collaboration, secure architectures and workflows, and effective evaluation frameworks for LM-based agents. This survey provides a unified overview of security risks in LM-based agents and guides future research toward secure, safe, and trustworthy agentic AI ecosystems.