Prompt engineering that survives production
Prompts that work in a playground break under real traffic. How to write, version and test them like the production code they are.
A prompt is not a note to a colleague. It is a configuration file that controls a probabilistic system, deployed to production, that changes behaviour when its dependencies update. Treat it accordingly and most prompt problems disappear.
Put prompts in version control, not in the database
Prompts stored in an admin panel feel flexible and become untraceable. When output quality changes, you want a diff and a commit author. Ship prompts with the code, review them in pull requests, and roll them back the same way you roll back anything else.
Be explicit about the failure case
Most hallucination is a specification gap. If you have not told the model what to do when the answer is not in the provided context, it will improvise, because that is what it is for. Say it plainly: if the context does not contain the answer, say so and stop.
Specification beats persuasion
Framing
- Does little
- Elaborate persona and role-play
- Does the work
- Clear task definition and constraints
Grounding
- Does little
- "Be accurate"
- Does the work
- "If the context lacks it, say so and stop"
Format
- Does little
- "Respond in JSON"
- Does the work
- Schema enforcement at the API
Structure
- Does little
- One long paragraph
- Does the work
- Delimited sections, instructions before content
| Does little | Does the work | |
|---|---|---|
| Framing | Elaborate persona and role-play | Clear task definition and constraints |
| Grounding | "Be accurate" | "If the context lacks it, say so and stop" |
| Format | "Respond in JSON" | Schema enforcement at the API |
| Structure | One long paragraph | Delimited sections, instructions before content |
Structure beats politeness
Delimit sections clearly, put instructions before content, and ask for structured output when you intend to parse it. Elaborate persona framing is mostly decoration; clear task definition, explicit constraints and worked examples do the actual work.
Test prompts like code
Every prompt change should run against a fixed set of cases with expected properties, scored automatically. Without that, prompt iteration is a random walk: you fix the case in front of you and silently break three you were not looking at.
- A golden set covering typical, edge and adversarial inputs
- Assertions on properties — cites a source, refuses when unsupported, returns valid JSON
- A regression run on every change, with the diff visible in review
- Sampled production traffic reviewed weekly to catch drift the set misses
Treating a prompt like production code
- 1
Version it
In the repository, reviewed in a pull request — not in an admin panel where changes are untraceable.
- 2
Specify the failure case
State what to do when the answer isn't in the context. Unspecified means improvised.
- 3
Test on every change
A golden set with assertions, run automatically. Without it, fixing one case silently breaks three.
- 4
Re-run when the provider ships
Models update under stable names. Your evals are how you find out what changed.
Expect the ground to move
Providers update models under stable names. A prompt tuned tightly to one version's quirks is a liability. Prefer clear instructions over exploits, pin versions where you can, and re-run your evals whenever a provider ships anything.