# Pm AI Context To Contract > Turn an AI or agent context change into a source-bounded context contract covering instructions, knowledge, tools, memory, state, query, selection rules, freshness, privacy, token budget, compaction, evaluation slices, fallback, and a ship, hold, or rollback decision. Use when a PM reviews prompts, retrieval, tool schemas, MCP, conversation history, memory, context assembly, or long-running agent behavior. - Skill: `asdc163/pm-ai-context-to-contract` (Agent Skill, multi-file: 4 files) - Install (CLI): `npx skillmds@latest add asdc163/pm-ai-context-to-contract` - Raw SKILL.md: https://api.skillmd.com/api/skills/asdc163/pm-ai-context-to-contract/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: asdc163 (https://skillmd.com/u/asdc163) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/asdc163/pm-ai-context-to-contract --- # PM AI Context to Contract Use this skill when an AI or agent may be seeing too much, too little, the wrong thing, or an untrusted thing for a user job. It turns a context change into a reviewable contract: what enters the model's working context, why it is there, how it is selected, which source wins when facts conflict, what must be removed or redacted, and how the team will know the change helped. The output is a product decision packet, not a prompt rewrite, context-builder implementation, model benchmark, provider recommendation, or production guarantee. Context is treated as a changing system boundary rather than a single prompt string. ## When to use Use it when: - an agent change adds or removes system instructions, retrieved knowledge, tools, MCP servers, conversation history, memory, task state, or examples; - a result changes after context grows, a session crosses a compaction point, a tool returns verbose data, or a source becomes stale; - a team needs to decide between preloaded context and just-in-time retrieval, raw history and a structured summary, or a full tool set and a task subset; - a user-facing AI feature needs source visibility, data-use boundaries, uncertainty, human control, or a safe fallback; - a context optimization reduces tokens or latency but its quality, privacy, trust, or task-completion effect is not yet known; - a long-running agent needs a handoff capsule or durable state instead of depending on a hidden conversation window. Use `pm-ai-evaluation-plan` when the primary work is a quality rubric and test dataset. Use `pm-ai-cost-to-guardrail` when the primary decision is economics or latency per successful outcome. Use `pm-ai-trace-to-regression` when one concrete run already failed. Use `pm-ai-task-boundary` when the open question is what a person versus an AI should own. Use `pm-decision-to-spec` after the context decision needs a broader product and UX handoff. Do not use this skill to expose hidden chain-of-thought, read private customer content, install a retrieval or memory system, call a model or provider, change a tool permission, choose a vendor, or claim that more context is better without outcome evidence. ## Guardrails 1. Frame one user job, success oracle, decision owner, observation window, and context change. Do not review a context bundle without naming the outcome it is meant to support. 2. Inventory six components separately: `instructions`, `knowledge`, `tools`, `memory`, `state`, and `query`. Do not hide state in memory or treat tool descriptions and tool results as harmless metadata; both enter context. 3. For every source record authority, provenance, freshness, version, privacy class, tenant or access boundary, selection rule, and evidence status. If a field is missing, write `Not provided` rather than guessing. 4. Declare inclusion and exclusion rules. A source that is relevant in theory may still be omitted because it is stale, redundant, unauthorized, untrusted, too large, or outside the job scope. 5. Set a proposed input budget by component and reserve room for the query, output, tool results, retries, and safe fallback. Never hard-code a provider context limit as a universal product truth. 6. Make conflict resolution explicit. A newer document does not automatically outrank a policy, and a retrieved paragraph does not become an instruction merely because it is inside the context. 7. Treat retrieved pages, files, tool results, issue comments, and memory as potentially untrusted data. Prompt injection, hidden instructions, cross-tenant content, secrets, and unsafe links are release blockers when they can change behavior or leak data. 8. Keep user-visible context visibility proportional to risk: show source identity, time or freshness, tool activity, limitations, and controls when useful; never reveal private raw content or hidden chain-of-thought as a trust substitute. 9. Test context changes with the same success oracle, relevant slices, a negative set, and component ablations. Fewer tokens or a prettier answer is not proof of better context. 10. Keep raw evidence separate from distilled context. A summary can guide the next call, but high-risk decisions need a path back to the original source or trace. 11. Record sanitized context-manifest fields for diagnosis: component IDs, source IDs, versions, counts, token ranges, selection reasons, truncation, retrieval, compaction, fallback, and outcome. Do not store raw prompts, secrets, customer text, or private URLs in a general analytics packet. 12. Keep a reversible route visible: restore the last known-good context version, narrow the source set, disable a memory write, expose fewer tools, ask for clarification, or hand the job to a person. ## Core definitions | Term | Meaning | Evidence status | | --- | --- | --- | | Context | The information available to the model for one inference or agent step | Declared boundary | | Instructions | System, developer, policy, role, format, and tool-use guidance | Source-bounded | | Knowledge | Retrieved documents, records, search results, or reference facts | Source-bounded | | Tools | Tool schemas, descriptions, permissions, and tool results | Separate from knowledge | | Memory | Durable or semi-durable user, project, or agent notes carried across calls | Requires write policy | | State | Current task progress, world state, approvals, timestamps, and pending work | Versioned runtime state | | Query | The user's current request and explicit parameters | User input | | Context contract | The inclusion, exclusion, authority, freshness, privacy, budget, and fallback rules for these components | Proposed or approved | | Context manifest | Sanitized evidence of what was selected for one run or sample | Observed only if instrumented | | Compaction | A transformation that replaces older context with a smaller continuation state | Must preserve critical facts | | Just-in-time context | A reference or tool path that retrieves a needed source during the run instead of preloading it | Latency and access tradeoff | Use these calculations only after scope is declared: ```text available_context_budget = declared_input_budget - reserved_output_and_tool_overhead component_budget_share = selected_component_budget / available_context_budget context_retention_rate = critical_facts_preserved_after_compaction / critical_facts_before_compaction ``` If the denominator, critical-fact set, input budget, or version boundary is missing, write `Not measurable`. Do not infer context quality from token count. ## Workflow ### 1. Frame the decision and user outcome Write one sentence: > We need to decide whether `...` context change can support the user job > `...` within `...` trust, privacy, quality, latency, and cost boundaries. Name the current workaround, decision owner, affected journey, success oracle, observation window, baseline context version, proposed context version, and the evidence that could change the decision. ### 2. Map the six context components Create one row for each source or source family. At minimum record: | Field | Question | | --- | --- | | Component | Is it instructions, knowledge, tools, memory, state, or query? | | Source ID | Can a reviewer identify the source without seeing raw private content? | | Purpose | Which user-job step does it support? | | Selection | Why is it included for this task and excluded for another? | | Authority | Is it policy, reference, user-provided, tool output, or an untrusted signal? | | Freshness | What time, version, TTL, or invalidation rule applies? | | Privacy | What data class, tenant, permission, or redaction boundary applies? | | Budget | What token, row, tool, or latency allowance is proposed? | | Omission risk | What fails if this source is absent or truncated? | | Fallback | What happens when the source is unavailable, stale, or conflicted? | Keep `memory` separate from `state`: a preference or durable note is not the same thing as the current approval, task checkpoint, or world state. ### 3. Register source authority, freshness, and trust Build a source ledger with IDs such as `S-001`, `T-004`, `M-002`, and `Q-001`. For every row record source, owner, retrieval or write time, version, access scope, effective window, and evidence status. Label user text, retrieved text, tool output, and memory as data unless the product has an explicit rule that promotes a field to an instruction. Write a conflict rule before reading the result. For example: current approved policy outranks an old article; a user claim can describe intent but cannot override a system safety rule; a tool result can supply data but cannot change its own permission. These are examples, not universal defaults. ### 4. Set the budget and selection policy Declare an input budget with a safety margin and reserve room for the query, expected output, tool responses, retries, and fallback. Allocate budget by component, then define what happens when a component exceeds it: - retrieve fewer, more diverse, higher-signal sources; - preserve identifiers and citations while dropping redundant prose; - expose a task-specific subset of tools; - compact history into a versioned handoff capsule; - move durable state to an approved artifact and load it just in time; - stop and ask for clarification instead of silently truncating critical rules. Do not turn a budget into a provider recommendation. The PM decision is which information the job needs and what can safely be omitted. ### 5. Design dynamic context and long-horizon continuity Choose among preloaded, just-in-time, hybrid, compacted, cached, or artifact-backed context. Record the tradeoff between freshness, latency, access control, retrieval failure, and observability. A compaction rule must state what it preserves: goals, decisions, constraints, source IDs, unresolved risks, approvals, and next action. If memory can be written, define who or what may write it, the validation gate, retention or deletion path, provenance, and how a user can correct or reset it. If a session crosses a context version, start a new trace segment instead of blending old and new state. ### 6. Map failure, privacy, and recovery states At minimum cover: ```text empty context -> source unavailable -> stale source -> conflicting source -> oversized context -> compaction -> untrusted tool/retrieval result -> privacy or permission mismatch -> safe fallback or human handoff ``` For each state define what the user sees, what the system records, what is blocked, how to recover, and whether the result can count as a completed job. Do not hide an omission, silent truncation, cross-tenant mismatch, or memory write behind a successful-looking final answer. ### 7. Define visibility and human control For a user-facing flow, specify the minimum useful evidence: source identity, freshness, selected tool or data category, missing-context warning, uncertainty, edit or reset control, approval boundary, and manual fallback. Show an activity or evidence summary rather than private reasoning traces. High-impact or irreversible actions remain subject to the approval contract in `pm-ai-approval-to-flow`. ### 8. Evaluate the context change Reuse the same completion oracle across baseline and candidate. Include: - representative user-job slices and hard negative or no-context cases; - stale, conflicting, unauthorized, injected, and malformed sources; - context-component ablations such as no memory, no retrieval, fewer tools, compacted history, or just-in-time retrieval; - source visibility, faithfulness, unsupported-claim, privacy, tool-routing, fallback, latency, and cost checks when applicable; - a small human review set when the failure cannot be safely judged by a deterministic check. Pre-commit `Ship`, `Iterate`, `Hold`, `Rollback`, or `Need evidence` rules. Keep component scores and context-manifest evidence separate from the final answer score. ### 9. Instrument and write back Specify safe fields for each run: trace ID, context version, component IDs, source versions, selected counts, token ranges, retrieval or tool events, memory read/write, compaction, truncation, access decision, fallback, outcome oracle, privacy class, and latency. Link a failure to the appropriate regression, incident, evaluation, cost, or release-learning surface. Keep the raw artifact under its approved access boundary. ### 10. End with one review ask Choose exactly one decision and name the owner, unresolved context risk, and next evidence that could change it. ## Output contract Return these sections in this order. Keep unsupported fields explicitly `Not provided`, `Not measurable`, `Not measured`, `Proposed`, `Not run`, `Unknown`, or `Not covered`. ## Decision on the desk State the decision, owner, user job, outcome, baseline and candidate context versions, proposed budgets, and the evidence boundary. ## User job and context outcome Define the success oracle, eligible job, current workaround, affected journey, and what a completed outcome means. Do not use token count as the outcome. ## Context inventory List instructions, knowledge, tools, memory, state, and query with source IDs, purpose, selection rule, authority, privacy, freshness, budget, omission risk, and fallback. ## Source, freshness, and trust contract Show provenance, version, effective window, conflict rule, access boundary, untrusted-data handling, and what is `Observed`, `Estimated`, `Proposed`, or `Not measured`. ## Budget and selection rules Show total input budget, reserved output/tool/fallback room, component shares, ranking or filtering rules, exclusion rules, truncation behavior, and the rollback path when the budget is exceeded. ## Dynamic state, memory, and compaction Describe preloaded versus just-in-time sources, state versus memory, write and reset policy, handoff capsule, compaction preservation set, and version boundary. ## Failure, privacy, and recovery Cover empty, stale, conflicting, injected, oversized, unauthorized, malformed, and unavailable contexts; state user-visible behavior, safe records, recovery, and human fallback. ## Evaluation and release gate List baseline/candidate, slices, negative cases, component ablations, oracle, quality/trust/privacy/tool/fallback gates, p50/p95 latency or cost links when relevant, and the predeclared decision rule. ## Instrumentation and writeback List the privacy-safe manifest and trace fields, owner, QA method, retention, and where failures or learnings become regressions, incidents, eval cases, cost guardrails, or release observations. ## Not covered Name every unexecuted provider call, raw-data review, user session, retrieval quality test, context-window measurement, model comparison, deployment, adoption, traffic, ROI, safety, or star claim. ## Review ask End with one of `Ship`, `Iterate`, `Hold`, `Rollback`, or `Need evidence`, one owner, and one next evidence request. ## Edge cases - **Only a token count exists:** record a budget signal, not a quality result; add a completion oracle and context-manifest evidence. - **The context window is large:** still set a product budget; larger capacity does not prove that redundant or stale context improves the job. - **Average latency is fine but retrieval is slow:** inspect p95 and the fallback boundary; separate source freshness from response speed. - **A source is relevant but stale:** show its age, block or downgrade it by policy, and preserve a safe fallback; never silently blend it with current policy. - **Instructions conflict:** use the declared authority rule and make the conflict a test case; do not let source order become an accidental policy. - **Tool or retrieval output contains instructions:** treat it as untrusted data until a separate policy explicitly promotes a field; check for prompt injection and unsafe tool chaining. - **Memory contains a wrong fact:** quarantine or delete the memory entry, record provenance, and test a clean session; do not ask the model to simply ignore it without a reset path. - **Compaction drops a critical decision:** hold or rollback the change; add a preservation test for goals, approvals, constraints, source IDs, and risks. - **Too many tools are exposed:** narrow the task-specific subset and measure tool selection, fallback, and negative routing; do not hide the change in a prompt rewrite. - **Silent truncation occurs:** make the boundary observable and fail safe; never count a truncated run as equivalent to the baseline. - **Cross-tenant or permission mismatch:** block the run, preserve only safe metadata, and route to security or incident handling. - **Just-in-time retrieval is fresher but slower:** compare p95, abandonment, source correctness, and fallback together; do not optimize one signal alone. - **Context reduction lowers cost but quality is unknown:** use `pm-ai-cost-to-guardrail` for the economics and keep this skill at `Hold` or `Need evidence` until the same outcome oracle is tested. - **Small or synthetic sample:** label it as directional or fictional; do not infer production reliability, adoption, or star impact. - **Raw evidence is too sensitive:** store approved source IDs, categories, hashes, or redacted excerpts only, with an access-controlled path to the original when required. ## Final check Before returning the contract, confirm: - the user job, outcome oracle, owner, baseline, candidate, and observation window are explicit; - all six context components are considered, including absent components; - each source has provenance, authority, freshness, version, privacy, access, selection, budget, omission risk, and fallback status; - memory and state are not blended, and tool schemas/results are visible in the inventory; - inclusion, exclusion, conflict, injection, and truncation rules are written before interpreting results; - the budget reserves room for output, tool responses, retries, and fallback; - compaction, just-in-time retrieval, caching, or artifacts have a stated freshness, latency, access, and recovery tradeoff; - user-facing visibility, uncertainty, correction/reset, approval, and manual fallback are covered when the job is high impact; - baseline/candidate tests use the same oracle, include negative cases and component ablations, and separate quality from token or latency signals; - manifest and trace fields are privacy-safe and can locate a failing layer; - fictional, synthetic, internal, observed, proposed, and production evidence are labelled; - `Not covered` contains every unexecuted claim and unsupported adoption, traffic, ROI, safety, or star conclusion; - the final line contains one decision, one owner, and one next evidence ask.