Learning by trial and consequence.
WHAT IT IS
A system takes actions in an environment and receives rewards or penalties. It learns which actions produce reward.
No labelled examples. Only outcomes.
THE ANALOGY
Training an animal with rewards. The system tries things and learns what works.
WHERE IT HAS SUCCEEDED
Games, where the rules are clear and the system can play millions of times Robotics Resource allocation and scheduling Recommendation systems optimising engagement
WHAT IT REQUIRES
An environment where the system can try things repeatedly and cheaply.
That is why games worked first: a simulation costs nothing per attempt.
THE DIFFICULTY IN THE REAL WORLD
Most business decisions cannot be tried a million times. Each attempt is expensive and consequential.
THE REWARD PROBLEM
The system optimises exactly what you reward, including in ways you did not intend.
Rewarding engagement produces engagement, not necessarily value.
RELEVANCE TO LANGUAGE MODELS
Reinforcement learning from human feedback is used to make models more helpful and better aligned with what people want.
