The principle.
WHAT AI CANNOT DO
Be accountable Notice that it is wrong Understand consequences Exercise judgement about an exception
WHAT THIS MEANS
A person must remain responsible for anything consequential.
WHERE OVERSIGHT IS ESSENTIAL
Decisions affecting people Anything published Anything a customer relies on Financial transactions Safety-related matters
WHAT MEANINGFUL OVERSIGHT LOOKS LIKE
A person who reads the output Who has the knowledge to evaluate it Who has authority to reject it Who has time to do so properly
THAT LAST POINT
Oversight that is nominal is not oversight.
A person rubber-stamping a hundred outputs an hour is providing no protection.
WHAT UNDERMINES IT
- Automation bias: people accept suggestions from systems
- Volume: too much to review properly
- Familiarity: verification decays over time
WHAT TO BUILD IN
Enough time to review Sampling where full review is impractical Periodic checks that review is happening
