START HERE
What you will learn
- Recognise the difference between learning a useful pattern and fitting a particular sample.
- Separate examples used for improvement from examples used for assessment.
- Interpret a comparison without treating a small test as proof of future performance.
Before you start: The neural-network guide introduces training. No statistical formulas are needed here.
Learning the pattern or learning the examples?
Imagine preparing for an exam by memorising the answers to one practice paper. You may score very well when the same questions return and struggle with unfamiliar ones. A model can show a comparable problem: it fits details of its training sample that are not dependable patterns in future data.
Training performance tells you how the model behaves on data used to fit it. To assess generalisation, examine other examples that represent the intended use. The comparison is particularly important when the model is complex, the sample is small or the training examples do not cover the situations you will encounter.
Keep improvement and assessment separate
Training data is used to fit parameters. Validation data helps choose settings or compare candidate approaches. A held-out test set is reserved for a later assessment after those decisions. If you repeatedly change the system after seeing the test results, the test is no longer an independent check of those changes.
A similar evaluation mistake can happen without neural-network training: repeatedly adjusting a prompt to pass the same five demo questions can make the demonstration look stronger than performance on fresh questions. That does not mean the prompt has trained the model’s parameters; it means you have selected a workflow around the examples you keep inspecting.
Step-by-step process
Define the future situations that matter
List the kinds of inputs, sources and users the system is supposed to handle. An evaluation made only of easy examples cannot tell you much about difficult work.
Separate the examples before improving the system
Keep development examples and later assessment examples distinct. Avoid near-duplicates that make the held-out work effectively familiar.
Record both development and held-out results
A large gap is a reason to investigate. Also inspect whether failures concentrate on a particular category or condition.
Prefer evidence over the best demonstration
Compare approaches on the same held-out cases using the same checks. A simpler approach may generalise better even if its training score looks less impressive.
Monitor genuinely new work
After release, review appropriate samples as inputs and documents change. New conditions can cause problems even for a system that previously tested well.
FOLLOW ONE EXAMPLE
Worked scenario
Two models sorting support tickets
These fictional numbers illustrate a comparison. Both models were fitted using 200 labelled tickets and are assessed on the same 50 separate tickets. Each ticket has an agreed correct destination.
- Model A sorts 196 of 200 training tickets correctly: 98%. On the separate tickets it sorts 35 of 50 correctly: 70%.
- Model B sorts 180 of 200 training tickets correctly: 90%. On the separate tickets it sorts 42 of 50 correctly: 84%.
- Model B has stronger results on this held-out sample despite its lower training score. Inspect the actual errors and whether the 50 tickets represent the intended workload before selecting it.
What this shows: Training score alone would favour A. The held-out comparison gives a different and more relevant signal, while still leaving uncertainty about future work.
The process at a glance
Read Mermaid diagram source
flowchart LR A[Representative examples] --> B[Training or development cases] A --> C[Reserved assessment cases] B --> D[Improve the system] D --> E[Assess on reserved cases] C --> E E --> F[Inspect errors and limits]
What you should have at the end
- A clear separation between improvement examples and assessment examples.
- Results that show where a model succeeds and fails on unfamiliar work.
- A more cautious and useful interpretation of a successful demo.
Common mistakes and how to avoid them
Optimising against the final test repeatedly
Once its answers guide your changes, it is no longer a fresh check of those changes.
Letting near-duplicates cross the boundary
Repeated customers, templates or almost identical records can make an assessment misleading.
Ignoring changes after launch
A new source format or different workload can invalidate earlier assumptions.
APPLY THE IDEA
Try it yourself
Your task: Take ten example questions for an AI task. Set aside three before changing any instructions. Improve the system using only the other seven.
What to produce: A note comparing the two groups and describing where the reserved examples differ.
Check your understanding: Did anything learned from the reserved examples influence your changes? If so, reserve fresh cases before making a final performance claim.
Key terms in plain English
- Overfitting
- Fitting the development or training examples more closely than is useful for new examples.
- Generalisation
- Performance beyond the examples used to fit the model.
- Held-out set
- Examples reserved from the improvement process for assessment.
- Data leakage
- Information crossing an evaluation boundary and making results misleadingly optimistic.