The system behind AI chat tools.
WHAT IT DOES
Given text, it predicts what text should come next.
Repeated, that produces paragraphs, articles, code and conversation.
HOW IT LEARNED
Trained on very large amounts of text, learning statistical patterns of how language works.
WHY THAT PRODUCES USEFUL BEHAVIOUR
To predict text well across an enormous variety of material, a system must capture a great deal about grammar, facts, reasoning patterns and style.
Capability emerges from the prediction task rather than being programmed.
WHAT IT IS NOT
A database. It does not look anything up. A reasoning engine. Its reasoning is pattern-matching that frequently resembles reasoning and sometimes fails oddly.
WHY IT HALLUCINATES
It predicts plausible text. Where it lacks information, plausible text is still produced.
WHAT MAKES MODERN ONES USEFUL
Additional training to follow instructions and be helpful, beyond raw text prediction.
THE PRACTICAL SUMMARY
Excellent at language. Unreliable about facts. Useful when you supply the facts.
