The system behind AI chat tools.
THE DEFINITION
A model trained on very large amounts of text to predict what text comes next.
Repeated, that produces paragraphs, code and conversation.
WHY PREDICTION PRODUCES USEFUL BEHAVIOUR
To predict text well across enormous variety, a system must capture a great deal about grammar, facts, reasoning patterns and style.
Capability emerges from the task rather than being programmed.
WHAT IT IS NOT
A database. It does not look anything up. A reasoning engine in any reliable sense.
WHY IT HALLUCINATES
It predicts plausible text. Where it lacks information, plausible text is still produced.
WHAT MAKES MODERN ONES USEFUL
Additional training to follow instructions and be helpful.
THE PRACTICAL SUMMARY
Excellent at language. Unreliable about facts. Useful when you supply the facts.
RELATED TERMS
Transformer, token, hallucination, context window.
