Glossary

Action Generator An action generator is a component in an agentic AI system responsible for deciding or proposing the next action to take based on the model's current goals, context, or environment.If compromised, it can be used to cause the system to behave erratically, pursue harmful objectives, or leak information.In adversarial contexts, attackers may try to hijack or bias the action generator to alter system behavior.Adversarial Dataset Optimization Adversarial dataset optimization is the process of selecting or engineering data samples that will make a machine learning model more vulnerable to attack.By systematically introducing subtle, harmful examples into the training data, attackers can maximize their influence on model predictions.This technique is often used to boost the effectiveness of poisoning and backdoor attacks.Adversarial Embedding An adversarial embedding is a data representation-often a hidden layer vector or word embedding-that is carefully crafted to trigger unwanted or malicious behavior in downstream models.It is particularly effective when the same embedding space is shared across models or systems, making it a stealthy attack vector.These manipulations can persist across tasks and transfer between applications.Adversarial Fine-Tuning Adversarial fine-tuning occurs when a machine learning model is deliberately retrained with malicious data or objectives that embed hidden or harmful behaviors.These behaviors are often subtle and may remain dormant unless specific triggers are activated.This type of attack is especially dangerous when using publicly shared model checkpoints or third-party fine-tuned models.Adversarial Input An adversarial input is any data sample intentionally designed to deceive or confuse a machine learning model during inference.These inputs may appear normal to humans but exploit edge cases or blind spots in the model's decision boundaries.Adversarial inputs are a foundational tool in evasion attacks, enabling attackers to manipulate outputs with minimal changes to the data.Adversarial Optimization Adversarial optimization refers to the use of search techniques-such as gradient descent, evolutionary strategies, or reinforcement learning-to find inputs, model parameters, or triggers that break or exploit an AI system.These methods are used to automate the creation of high-impact attacks.It is a core process behind adversarial training, perturbation crafting, and model evasion design.