Systems handling more than text.
WHAT IT MEANS
A model working with several kinds of input or output: text, images, audio, video.
WHAT THIS ENABLES
Describing an image Answering questions about a photograph, a chart or a screenshot Extracting text and data from documents Generating images from descriptions Transcribing and understanding speech
PRACTICAL USES
Reading a receipt or invoice and extracting the figures Explaining a diagram Describing a photograph for accessibility Diagnosing a problem from a screenshot Converting a handwritten note to text
WHAT IT DOES WELL
Describing what is visibly present Extracting text Answering questions about clear images
WHAT IT DOES BADLY
Precise measurement or counting Reading poor-quality images reliably Anything requiring specialist visual judgement
THE CAUTION
Do not rely on it for readings where an error matters: medical images, engineering measurements, financial figures from poor scans.
Verify extracted data against the source.
WHERE IT IS GENUINELY USEFUL
Turning visual material into text you can then work with.
