Overview
A prompt that has been in production for a while accumulates rules and examples one bug fix at a time. The mix drifts: examples pile up to patch edge cases a single rule would cover, or a rule grows into a paragraph that is really a demonstration. This page covers the maintenance side of examples-vs-rules: the signals that a constraint is in the wrong form, how to refactor it, and how to confirm with evals that the new mix is better. For choosing and ordering individual examples, see few-shot.
Examples decay; rules compound
Adding a tenth example to a few-shot set usually does little; the marginal gain is small. Adding a tenth rule to a system prompt can change behavior across the entire input distribution.
- If you find yourself adding more examples to fix more edge cases, ask whether a rule would cover them at once.
- If you find yourself adding more rules and the model keeps drifting on shape, ask whether one example would pin the shape.
Track the trade in the eval set. A rule that lifts the adversarial slice by 5 points is worth more than an example that lifts easy by 1 point.
When the rule is too long, refactor to an example
A 200-word rule is often a sign the rule is actually a demonstration in disguise. If you cannot state the rule in one sentence, try replacing it with an example.
- “Always format dates as YYYY-MM-DD, with a hyphen, four-digit year, etc., except in fiscal year contexts where…” becomes one example showing the right format in each context.
- “Always reply in the user’s voice, matching their tone, brevity, and formality” becomes three examples of the model matching three different user voices.
Examples carry constraints that resist explicit statement. Rules carry constraints that benefit from explicit statement. Pick by which the constraint is.
Test both with the same eval suite
The right mix is empirical. Run the eval set with rules-only, examples-only, and combined; pick the configuration that wins on the slices that matter.
- A rules-only baseline tells you how much the examples actually contribute.
- An examples-only baseline tells you how much the rules contribute.
- The combined version usually wins, but the lift over the better of the two singles tells you whether the examples earn their token cost.
See evaluation for the eval pattern. See role-framing for the role rules that sit alongside both.