Improper Output Handling

Exposure

Improper output handling occurs when the text, code, or data produced by a language model is consumed by downstream systems — web renderers, database interpreters, command shells, or file systems — without the same validation applied to user-supplied input.

The model is trusted as a source, so its output bypasses input validation controls. An attacker who controls what the model reads (through prompt injection, data poisoning, or retrieved content) can therefore indirectly control what downstream systems execute.

Signals

• Model output is rendered as HTML, executed as code, or used in database queries without sanitisation.
• The application treats model-generated content as inherently safe.
• Downstream systems lack input validation at the boundary where AI output enters.
• Plugin or tool outputs are concatenated into system commands without escaping.

Failure pattern

A user asks the model a benign question. The model retrieves a web page containing hidden JavaScript or SQL fragments. The model reproduces those fragments in its response. The application renders the response directly in a browser or passes it to a database, executing the attacker’s payload through the AI system’s trusted channel.

Related controls

• Validate and sanitise all model output before it reaches downstream systems.
• Apply context-appropriate encoding: HTML-encode before rendering, parameterise before querying.
• Treat model-generated content as untrusted input at every system boundary.
• Log output flows to detect anomalies in format or content.