Risk
Misinformation
The model generates false or misleading information presented with apparent authority, which users may accept without verification, with legal, reputational, or security consequences.
- Domain
- Model
- Human / Organizational
- Mechanism
- Integrity
- Overreliance
- Boundary
- Between model output and user or downstream acceptance
- Trigger
- The model generates false or misleading content with apparent authority
- Consequence
- Decisions taken on unverified content; reputational or legal harm
- Required evidence
- Verification workflows and fact-check records
- Confidence calibration samples
- Content review logs
Exposure
Misinformation from language models is not merely an accuracy problem — it becomes a security concern when users or downstream systems act on fabricated information. The model’s output style — confident, articulate, well-structured — lends credibility to content that may be entirely false.
The risk compounds when model output feeds into automated decision pipelines, customer-facing applications, medical or legal advisory systems, or public communication channels where the cost of correction is high and the window for verification is narrow.
Signals
- Model generates citations, statistics, or technical details that cannot be verified against source material.
- Users or downstream systems treat model output as authoritative without independent verification.
- Automated workflows execute decisions based on unverified model-generated conclusions.
- Content produced by the model is published or distributed before human review.
Failure pattern
A user queries the model for factual information. The model, lacking grounding in verifiable sources, generates a plausible-sounding but fabricated response complete with invented citations, statistics, and technical terminology. The user accepts the output as accurate. In a security context, this could lead to incorrect risk assessments, flawed incident response decisions, or compliance submissions based on non-existent evidence.
Related controls
- Ground model responses in verifiable, authoritative sources where accuracy is critical.
- Implement retrieval-augmented generation with auditable source attribution.
- Require human review for model output that informs high-stakes decisions.
- Display confidence indicators and source links alongside generated content.
- Log and monitor factuality complaints to identify systematic hallucination patterns.
Provenance
- SOURCE
-
- OWASP Top 10 for LLM Applications 2026 ↗ external_authority
- MAPPED
- Not recorded
- ANALYZED
- Not recorded
- UNRESOLVED
- Not recorded
Last reviewed: Sep 20, 2026