A language model predicts plausible text rather than retrieving verified facts, so when context is thin it invents details, nonexistent clauses, or made up figures in the same fluent tone as correct answers. Grounding lowers the rate but never eliminates it.
The new risk agents introduce
A chatbot's error stops at the screen, while an agent writes the fabrication into records, tickets, and downstream documents that other systems then treat as authoritative, and it steers its own next steps before any human sees it.
Contributing factors
- No ground truth exists inside the model, so it cannot tell accurate content from invented content.
- Fluent, confident phrasing hides errors and disarms reviewer skepticism.
- Ambiguous or incomplete prompts push the model to fill gaps with plausible inventions.
- Helpfulness tuning rewards giving an answer over admitting uncertainty.
- Retrieval gaps and stale knowledge bases leave questions the model answers from memory.
Examples
- A customer support assistant cites a refund policy clause that does not exist, and the wrong guidance is pasted into an internal procedure document.
- A clinical documentation agent invents a past procedure when summarizing a patient record, and the fabricated history is saved to the chart.
- A coding assistant references a library function that was never defined, and the call ships into a build before review catches it.