Exposure and failure modes
Observed exposure paths where generative AI connects language to data, identity, tools, permissions, or decisions before governance has enough grip.
Training data, fine-tuning datasets, or embeddings are deliberately manipulated to introduce vulnerabilities, backdoors, or biases that compromise model security and behaviour.
Model-generated output is passed to downstream systems without validation or sanitisation, enabling injection attacks through AI-generated content.
The model generates false or misleading information presented with apparent authority, which users may accept without verification, with legal, reputational, or security consequences.
Internal system instructions, business rules, or configuration not intended for end users are exposed through model responses, revealing logic and enabling targeted follow-on attacks.
The application permits uncontrolled resource usage or model queries — without rate, cost, or volume limits — enabling denial of service, financial exhaustion, or service abuse.
Systems that generate, store, or query vector embeddings in RAG architectures introduce their own security risks: injection through indexed data, cross-tenant information leakage, and retrieval manipulation.
The trust channel between an MCP client and server can carry malicious content, intercepted requests, or manipulated trust dialog, allowing an attacker to control agent behavior through the communication channel.
An attacker-controlled artifact reaches an AI coding agent through plugin resolution, symlink paths, model cache, or CI trigger, and executes with the agent's privileges.
An AI system can take actions that exceed its reliability, authorization, or oversight model.
AI systems call tools or APIs without sufficient validation, authorization, rate limits, or operational safeguards.
Untrusted instructions enter an AI workflow and compete with the system's intended authority.
Sensitive business, personal, regulated, or secret data moves through prompts, outputs, logs, retrieval, or connected tools.
Unapproved or unknown AI use removes visibility from data handling, identity, procurement, and incident response.
