Engineering around language models.
WHAT THE OPTIONS ARE, IN ORDER OF COST
Prompting a hosted model Retrieval-augmented prompting Fine-tuning a smaller model Training from scratch
WHAT TO TRY FIRST
Prompting, always.
WHY
It requires no training, and establishes whether the task is achievable at all.
WHAT RETRIEVAL-AUGMENTED GENERATION DOES
Retrieves relevant material and supplies it with the question.
WHY THAT IS USUALLY BETTER THAN FINE-TUNING FOR KNOWLEDGE
Knowledge changes, and retrieval updates instantly while fine-tuning does not.
WHAT FINE-TUNING SUITS
Format and style Task-specific behaviour Reducing cost by using a smaller model
WHAT IT DOES NOT SUIT
Teaching facts reliably.
WHAT PARAMETER-EFFICIENT FINE-TUNING PROVIDES
Adaptation by training a small number of additional parameters.
WHY THAT MATTERS
It makes fine-tuning affordable on modest hardware.
WHAT TO ESTABLISH BEFORE FINE-TUNING
That prompting genuinely cannot achieve it.
WHAT TO MEASURE
Cost per request, latency, and quality, together.
WHAT TO PLAN FOR
Model deprecation, and behaviour changing between versions.
WHAT TO NEVER DO
Build a system with no evaluation, then change the model.
