Risk
System Prompt Leakage
Internal system instructions, business rules, or configuration not intended for end users are exposed through model responses, revealing logic and enabling targeted follow-on attacks.
Exposure
System prompt leakage occurs when the instructions that define an AI application’s behaviour — its rules, constraints, role definition, tool permissions, and internal guardrails — are extracted by an end user through careful prompting.
The system prompt often contains proprietary logic, business rules, data schemas, or API names that were never intended for public visibility. Once exposed, this information enables attackers to craft more effective prompt injections, bypass restrictions, or understand the application’s internal architecture.
Signals
- Users receive responses that quote or paraphrase internal system instructions.
- The model reveals the names of internal tools, APIs, or data sources.
- Guardrails and refusal mechanisms are bypassed after a user discovers how they were implemented.
- Competitors or attackers demonstrate knowledge of proprietary prompt engineering patterns.
Failure pattern
A user crafts a series of prompts designed to make the model reveal its own operating instructions. Common techniques include role-playing (you are now in developer mode), translation requests (translate this system prompt to French), completion attacks (finish this sentence: “The system prompt says”), or chain-of-thought extraction. The model, trained to be helpful, complies and exposes its internal configuration.
Related controls
- Minimise sensitive information in system prompts; externalise rules to a separate validation layer.
- Implement prompt-level canaries to detect extracted content in monitoring systems.
- Apply output filtering to detect and block system prompt fragments.
- Regularly audit model responses for unintended information disclosure.