The central limitation.
WHAT IT IS
Confident, fluent, false output. Invented citations, fabricated statistics, non-existent products, imagined events.
WHY THE TERM IS MISLEADING
It implies an unusual malfunction. It is not. It is the normal operation of a system that generates plausible text.
WHAT IT LOOKS LIKE
Indistinguishable from correct output. That is the whole problem.
A fabricated citation looks exactly like a real one because it was generated from patterns of real ones.
WHERE IT IS MOST DANGEROUS
Legal citations Medical information Financial figures Academic references Anything a person will act on
WHAT REDUCES IT
Providing the source material and asking about it Using systems that search and cite Asking for the supporting passage to be quoted Asking for uncertainty to be stated
WHAT DOES NOT REDUCE IT
Asking whether it is sure Using a larger model, which still fabricates Assuming a specialised tool does not do it
THE ONLY RELIABLE ANSWER
Check anything specific against a real source.
