Confident false output.
THE DEFINITION
A model producing confident, fluent, false information: invented citations, fabricated statistics, non-existent products.
WHY THE TERM IS MISLEADING
It implies an unusual malfunction. It is the normal operation of a system that generates plausible text.
WHY IT HAPPENS
The model generates statistically likely text, not true text.
Where it lacks information, it produces something shaped like an answer.
WHY IT IS CONVINCING
Fluency and accuracy are separate properties. A fabricated citation looks exactly like a real one.
WHERE IT IS MOST DANGEROUS
Legal citations, medical information, financial figures, academic references.
WHAT REDUCES IT
Providing source material and asking about it Using systems that search and cite Asking for supporting passages to be quoted
WHAT DOES NOT
Asking whether it is sure.
RELATED TERMS
Confabulation, grounding, retrieval augmented generation.
