AI Behavior Contract
When To Use
"It should be helpful and safe" is the whole spec. The prompt says be polite and accurate, and the first time a user asks for a refund the assistant cannot grant, nobody knows whether it should decline, promise a callback or hand off. It answers: what must it always do, what must it never do whatever the input, what does it do when unsure, and how will each line be tested?
When Not To Use
If you need the instructions the model actually reads, run System Prompt Brief; the contract is the test that prompt must pass. If the open question is who takes over when the assistant is unsure, run Human-in-the-Loop Design.
Inputs
- The AI PRD behavior summary or AI Success Criteria, and the top failure modes from AI Failure Modes Map if you have it
- A handful of real inputs that went wrong or nearly did, with personal data masked
If you have none of this, I start from the feature's main job and its worst failure, and mark the output as a first draft.
Approach
Given-When-Then comes from behavior-driven development, as set out in the Cucumber Gherkin reference: a context, an event, an observable outcome. A model does not answer the same way twice, so each scenario here runs several trials, the idea of trials and of pass^k (every one of k runs passes) set out by Anthropic Engineering in "Demystifying evals for AI agents". The failure it prevents: a must-never line that held in the demo and broke on the fourth run, in front of a user.
Workflow
- Ask three questions: what is the one thing it must never do, what should it do when it does not know, and which inputs come from strangers (emails, tickets, web pages)?
- Write three lists in plain words: Must (always does), Must never (never does, whatever the input), When unsure (asks, offers options, declines, hands off, says it does not know). Keep each line to one behavior.
- Turn each line into a scenario: Given (context and state), When (input or event), Then (what an observer can check in the output). "Then it is helpful" is not checkable; "Then it names the order status and the next step" is.
- Use a Scenario Outline with an Examples table for variants: phrasings, languages, edge cases, a hostile instruction hidden in a pasted document.
- Set the bar per line as trials and a pass rate, both
[set by: name]. For must-never lines read pass^k: one failure in k runs fails the line, because the user who hits it does not see the average.
- List the guardrail behind each must-never outside the prompt (a filter, a permission, a human approval), since instructions in a prompt can be talked around. Tag every line with the eval check that will test it, to be written in Eval Rubric.
Output Format
# AI Behavior Contract
**Feature:** [name] | **Version:** [number] | **Owner:** [name]
## Must, must never, when unsure
| ID | Type | Line (one behavior) |
|---|---|---|
| M1 | Must | [line] |
| N1 | Must never | [line] |
| U1 | When unsure | [line] |
## Scenarios
**N1** Given [context and state] When [input or event] Then [observable outcome]
| Variant | Input example (masked) | Expected outcome |
|---|---|---|
| [phrasing, language, edge case] | [input] | [outcome] |
## Bar and checks
| ID | Trials | Pass rule | Bar | Guardrail outside the prompt | Eval check |
|---|---|---|---|---|---|
| N1 | [set by: name] | all k pass | [set by: name] | [filter, permission, approval] | [check ID, to write] |
## Decision
[Named person] sets the trials and pass rate for each line and approves the contract by [date].
Done When
- Every line holds one behavior, and every Then is something an observer can check
- Every must-never line reads pass^k and names a guardrail outside the prompt, or says none yet
- Every line carries a blank bar for a named person and a tag to an eval check
Quality Bar
- No pass rates, trial counts or results invented;
[not yet run] until a run is pasted
- Scenario inputs are realistic and masked, and synthetic variants are marked synthetic
- No line depends on the model "knowing" a rule that is not in its instructions or context
- Hostile input sits in the scenarios, not only polite users
- Claude writes the scenarios; a named person sets the pass rate each line must hold
Next
Run aipm-human-in-the-loop (Human-in-the-Loop Design) to design where a person steps in for the when-unsure lines.
About the makers
This pack is made by 90n (90n.ai): AI-native ways of working and tools for Product, Marketing and HR teams, built by ex-McKinsey founders. We map where time and money leak, redesign the work around AI and stay until your people run it themselves, and we run our own company on Claude. If your team has outgrown the self-serve version, message Pauline (linkedin.com/in/paulinebertry).
1---2name: aipm-behavior-contract3description: Writes an AI Behavior Contract of must, must never and when-unsure lines, each as a Given-When-Then scenario with a pass rate over repeated runs set by a named person, a guardrail list for what the prompt alone cannot enforce, and a tag to the eval that checks each line. Use for "run aipm-behavior-contract", "behavior spec for our AI", "must never rules for the assistant", "Given When Then for an LLM", "turn helpful and safe into tests", "what should the bot refuse", "guardrails spec", part of the Claude for AI Product Managers Pack by 90n.4metadata:5 source: https://skillmd.com/skills/polar-bear-org/aipm-behavior-contract6---78# AI Behavior Contract910## When To Use11"It should be helpful and safe" is the whole spec. The prompt says be polite and accurate, and the first time a user asks for a refund the assistant cannot grant, nobody knows whether it should decline, promise a callback or hand off. It answers: what must it always do, what must it never do whatever the input, what does it do when unsure, and how will each line be tested?1213## When Not To Use14If you need the instructions the model actually reads, run System Prompt Brief; the contract is the test that prompt must pass. If the open question is who takes over when the assistant is unsure, run Human-in-the-Loop Design.1516## Inputs17- The AI PRD behavior summary or AI Success Criteria, and the top failure modes from AI Failure Modes Map if you have it18- A handful of real inputs that went wrong or nearly did, with personal data masked19If you have none of this, I start from the feature's main job and its worst failure, and mark the output as a first draft.2021## Approach22Given-When-Then comes from behavior-driven development, as set out in the Cucumber Gherkin reference: a context, an event, an observable outcome. A model does not answer the same way twice, so each scenario here runs several trials, the idea of trials and of pass^k (every one of k runs passes) set out by Anthropic Engineering in "Demystifying evals for AI agents". The failure it prevents: a must-never line that held in the demo and broke on the fourth run, in front of a user.2324## Workflow251. Ask three questions: what is the one thing it must never do, what should it do when it does not know, and which inputs come from strangers (emails, tickets, web pages)?262. Write three lists in plain words: Must (always does), Must never (never does, whatever the input), When unsure (asks, offers options, declines, hands off, says it does not know). Keep each line to one behavior.273. Turn each line into a scenario: Given (context and state), When (input or event), Then (what an observer can check in the output). "Then it is helpful" is not checkable; "Then it names the order status and the next step" is.284. Use a Scenario Outline with an Examples table for variants: phrasings, languages, edge cases, a hostile instruction hidden in a pasted document.295. Set the bar per line as trials and a pass rate, both `[set by: name]`. For must-never lines read pass^k: one failure in k runs fails the line, because the user who hits it does not see the average.306. List the guardrail behind each must-never outside the prompt (a filter, a permission, a human approval), since instructions in a prompt can be talked around. Tag every line with the eval check that will test it, to be written in Eval Rubric.3132## Output Format33```markdown34# AI Behavior Contract35**Feature:** [name] | **Version:** [number] | **Owner:** [name]36## Must, must never, when unsure37| ID | Type | Line (one behavior) |38|---|---|---|39| M1 | Must | [line] |40| N1 | Must never | [line] |41| U1 | When unsure | [line] |42## Scenarios43**N1** Given [context and state] When [input or event] Then [observable outcome]44| Variant | Input example (masked) | Expected outcome |45|---|---|---|46| [phrasing, language, edge case] | [input] | [outcome] |47## Bar and checks48| ID | Trials | Pass rule | Bar | Guardrail outside the prompt | Eval check |49|---|---|---|---|---|---|50| N1 | [set by: name] | all k pass | [set by: name] | [filter, permission, approval] | [check ID, to write] |51## Decision52[Named person] sets the trials and pass rate for each line and approves the contract by [date].53```5455## Done When56- Every line holds one behavior, and every Then is something an observer can check57- Every must-never line reads pass^k and names a guardrail outside the prompt, or says none yet58- Every line carries a blank bar for a named person and a tag to an eval check5960## Quality Bar61- No pass rates, trial counts or results invented; `[not yet run]` until a run is pasted62- Scenario inputs are realistic and masked, and synthetic variants are marked synthetic63- No line depends on the model "knowing" a rule that is not in its instructions or context64- Hostile input sits in the scenarios, not only polite users65- Claude writes the scenarios; a named person sets the pass rate each line must hold6667## Next68Run aipm-human-in-the-loop (Human-in-the-Loop Design) to design where a person steps in for the when-unsure lines.6970## About the makers7172This pack is made by 90n ([90n.ai](https://90n.ai)): AI-native ways of working and tools for Product, Marketing and HR teams, built by ex-McKinsey founders. We map where time and money leak, redesign the work around AI and stay until your people run it themselves, and we run our own company on Claude. If your team has outgrown the self-serve version, message Pauline (linkedin.com/in/paulinebertry).