An AI assistant can produce a fluent, detailed answer that is wholly or partly untrue. The problem is not confined to obscure trivia: an invented court citation, a reversed research conclusion, or an unsupported statistic can change a real decision.
What a hallucination actually is
The NIST Generative AI Profile uses the term confabulation for erroneous or false generated content presented with apparent confidence. This is distinct from ordinary uncertainty about a difficult factual question. A system may compose a plausible response even when it lacks the necessary evidence, because fluent generation and verifying truth are different operations. The risk grows when users interpret polished prose as verified expertise.
Four common failure patterns
Fabrication: a report title, URL, quotation, number, or person is invented. Conflation: separate events or versions are merged. Temporal drift: a fact that was true in one year is treated as current. False justification: a persuasive explanation is appended to a wrong answer. A source citation may accompany any of these errors; the presence of a footnote is not a guarantee.
A verification routine for readers
Identify decision-bearing claims first: names, dates, legal requirements, prices, medical findings and numbers. Open the original record rather than relying on the AI’s summary of it. Verify that the cited passage supports the precise claim and whether it applies to the country, date, product version and population in question. Check the document’s issuance date against the event date. For disputed claims, use independent evidence with a different provenance chain. Record what remains unknown.
A safer workflow for teams
Separate draft generation from fact confirmation. Mark unverified assertions in a private editorial checklist, give reviewers access to primary records, and reject an answer if it mixes source text with fabricated details. Ask the system to abstain when evidence is missing, but test whether it actually abstains; merely adding “be accurate” to a prompt is not a control. Use a human accountable for release in consequential contexts.
What does not solve it
Low temperature, longer prompts, more confident wording and a search-enabled button may improve consistency or retrieval without guaranteeing correctness. Retrieval can bring stale or malicious text into the answer. A second model can repeat the first model’s mistake. Test the whole process using real representative questions, not isolated demonstrations.
Decision rule
For brainstorming and fiction, plausibility may be enough. For factual publishing, verify each consequential claim. For diagnosis, legal action, eligibility, physical safety or financial decisions, obtain qualified human evaluation and applicable primary records; do not let generated text become a unilateral decision.
Sources & Further Reading
- NIST AI 600-1 Generative AI Profile — NIST.
- NIST AI Risk Management Framework 1.0 — NIST.
- AI Playbook for the UK Government — UK Government.
Guidance varies by jurisdiction and application. This is general education, not legal, medical or other professional advice.