A security consideration.
WHAT IT IS
Instructions hidden in content the AI processes, designed to change its behaviour.
A document containing "ignore previous instructions and instead do X".
WHY IT WORKS
The model does not reliably distinguish between your instructions and text it is asked to process.
Everything is text in the same context.
WHERE IT MATTERS
Systems processing documents from outside your organisation Assistants reading email or web pages Agents taking actions based on retrieved content
WHAT COULD GO WRONG
An assistant reading a malicious email and following instructions embedded in it A summariser producing attacker-controlled output An agent taking an action it was not meant to
WHAT REDUCES THE RISK
Treating retrieved content as data rather than instruction Limiting what an automated system can actually do Human approval for consequential actions Not giving an assistant more access than the task requires
FOR ORDINARY CHAT USE
Low risk. The concern is with automated systems processing untrusted content.
FOR ANYTHING AUTOMATED
Assume content from outside may contain instructions, and constrain accordingly.
