Rudder Real Runtime Verifier Maintainer
Verify Rudder agent runtime behavior on the user's real local Rudder instance.
This is a black-box runtime acceptance workflow, not an implementation or code
review workflow.
Default to Chinese when the user asks in Chinese. Put the current truth first:
which runtimes passed, failed, or were blocked, and what transcript evidence
proves it.
Role Boundary
Default to verification and diagnosis only:
- Run real local runtime probes and inspect run transcripts, logs, API state,
issue state, comments, and run metadata.
- Create disposable orgs, agents, issues, and runs when needed for proof.
- Separate provider/model failure from Rudder adapter failure.
- Report exact blockers and smallest likely fixes.
- As an independent verifier, do not edit source, configs, Git state, or product
docs. Return findings to the implementer. The parent retains any existing
fix authority; verification does not require that authority to be repeated.
If the user asks to fix the issue, hand back to the lifecycle implementation
route or make the smallest explicit patch, then require this skill's real
runtime proof again before claiming done.
When This Skill Wins
Use this skill when the core question is whether an agent runtime actually did
the work through Rudder-managed tools in a real local run.
Typical prompts:
- "你所有的 agent runtime 都本地测过跑过真实环境了吗?"
- "OpenCode and Pi agent 你也测了吗?"
- "看 transcript,别让它 fallback 用 rudder cli"
- "排查所有 rudder tools,都试一遍"
- "MCP tool 报 org id/auth 问题,正常 agent 调 tool 不该传 org"
- "Codex/Claude/OpenCode/Pi 真实环境跑一下"
If the user asks for general product acceptance that is not runtime/tool-call
specific, use product-acceptance-verifier-maintainer instead. If the user
provides only one failed run id and wants root cause, use
debug-run-transcript-maintainer first, then return here for rerun proof after
a fix.
Runtime Matrix
Select the runtime matrix from the user's request and the changed integration.
Test one runtime for a runtime-specific claim. Use the full supported matrix for
an explicit all-runtime claim or a shared adapter/tool change that affects it:
Extend the matrix when the user names additional supported runtimes such as Cursor or
Gemini. Do not treat a Codex pass as proof for another runtime.
Read only the relevant reference files:
references/probe-workflow.md: real local setup, disposable data, probe
script expectations, and proof levels.
references/transcript-evidence.md: how to prove tool use and reject CLI
fallback from logs.
references/codex.md: Codex MCP transcript shape and known traps.
references/claude.md: Claude MCP transcript shape and strict config traps.
references/opencode.md: OpenCode MCP transcript shape, provider/model
caveats, JSONL parser expectations, and final-text pitfalls.
references/pi.md: Pi native extension transcript shape, schema/error
propagation traps, and 429/auth/provider separation.
references/reporting.md: verdict format, mutation ledger, and pass/fail
language.
Acceptance Standard
A runtime passes only when all are true:
- The local Rudder source of truth is identified, usually
GET /api/health on http://127.0.0.1:3100.
- A real local run was triggered for that runtime, or an existing real local
run was inspected with enough raw transcript/log evidence.
- The transcript shows the expected Rudder MCP/native tool calls.
- There is no model-visible fallback to shell, Bash, curl, or
rudder CLI for
Rudder work.
- Tool results are not internally failed (
isError, structured error,
rudder_cli_command_failed, missing required argument, auth/org failure).
- The terminal product effect was read back: issue status, comments, run
status, final text, API result, or another requested surface.
- Provider/model blockers are separated from Rudder adapter/tool blockers.
If the user requires "all tools", do not only run a happy-path issue workflow.
Run or request a manifest-driven coverage plan: list every exposed tool, verify
schemas load, check runtime-managed identity is not model-provided, and execute
representative read/mutate/file/image/pagination/error cases. Mark unexecuted
tools as not covered.
Default Probe Ladder
Use the smallest probe that answers the question, then escalate only as needed:
- Tool availability probe: call
rudder_agent_me, require final answer
to say MCP/tool path and no fallback.
- Issue workflow probe: seed an issue, then require
rudder_issue_context, rudder_issue_checkout,
rudder_issue_comment, and rudder_issue_done; read back final issue
status and comments.
- Representative matrix probe: add read/list/pagination/file/comment/image
examples such as
rudder_runs_errors, rudder_library_file_ref,
inbox/context, review, chat, or automation.
- Full manifest audit: prove every exposed Rudder tool has a stable name,
schema, description, and handler classification; execute safe read-only
tools and representative mutating tools in disposable data.
Known Judgment Rules
- A successful final answer is not enough. Inspect raw or parsed transcript.
- A tool call name in prompt text is not evidence. Use structured tool-call
events or parsed run-intelligence entries.
- A run can fail after completing the product effect. Report that separately:
tool/product effect passed; runtime finalization failed.
- A provider 429/auth/model error before any tool call is not an MCP auth/org
bug. Mark it
blocked_provider.
- If a tool result contains
isError: true but the runtime marks the tool call
completed, treat the runtime as failed or at least suspect until fixed.
- Internal runtime-owned bridges may execute a server process. That is
acceptable only if the model-visible action is a typed Rudder tool and
runtime env owns identity. It is not acceptable if the model uses Bash/curl or
runs
rudder ... itself for Rudder work.
Output
Use references/reporting.md for the full shape. At minimum include:
- overall verdict:
PASS, FAIL, or QUESTION for the requested matrix;
observed required behavior failure yields FAIL; otherwise missing required
proof or provider access yields QUESTION; complete required proof yields PASS
- verdict per runtime:
PASS, FAIL, BLOCKED_PROVIDER, PARTIAL, or
NOT_RUN
- run IDs, issue IDs, org IDs when disposable data was created
- observed tool names
- whether CLI/Bash/curl fallback was observed
- internal tool errors
- terminal product effect
- skipped coverage and why
- mutation ledger
Do not say "all green" if any runtime is blocked, substituted, partial, or only
covered by simple agent_me.
1---2name: rudder-real-runtime-verifier-maintainer3description: "Use when verifying Rudder agent runtime behavior in a real local environment, especially MCP/native Rudder tools across Codex, Claude, OpenCode, Pi, or user-named runtimes. Trigger for requests like 真是/真实环境跑过吗, 排查所有 rudder tools, transcript/fallback verification, runtime MCP availability, provider matrix proof, or checking whether agents used Rudder tools instead of rudder CLI/Bash/curl fallback."4metadata:5 source: https://skillmd.com/skills/undertone0809/rudder-real-runtime-verifier-maintainer6---78# Rudder Real Runtime Verifier Maintainer910Verify Rudder agent runtime behavior on the user's real local Rudder instance.11This is a black-box runtime acceptance workflow, not an implementation or code12review workflow.1314Default to Chinese when the user asks in Chinese. Put the current truth first:15which runtimes passed, failed, or were blocked, and what transcript evidence16proves it.1718## Role Boundary1920Default to verification and diagnosis only:2122- Run real local runtime probes and inspect run transcripts, logs, API state,23 issue state, comments, and run metadata.24- Create disposable orgs, agents, issues, and runs when needed for proof.25- Separate provider/model failure from Rudder adapter failure.26- Report exact blockers and smallest likely fixes.27- As an independent verifier, do not edit source, configs, Git state, or product28 docs. Return findings to the implementer. The parent retains any existing29 fix authority; verification does not require that authority to be repeated.3031If the user asks to fix the issue, hand back to the lifecycle implementation32route or make the smallest explicit patch, then require this skill's real33runtime proof again before claiming done.3435## When This Skill Wins3637Use this skill when the core question is whether an agent runtime actually did38the work through Rudder-managed tools in a real local run.3940Typical prompts:4142- "你所有的 agent runtime 都本地测过跑过真实环境了吗?"43- "OpenCode and Pi agent 你也测了吗?"44- "看 transcript,别让它 fallback 用 rudder cli"45- "排查所有 rudder tools,都试一遍"46- "MCP tool 报 org id/auth 问题,正常 agent 调 tool 不该传 org"47- "Codex/Claude/OpenCode/Pi 真实环境跑一下"4849If the user asks for general product acceptance that is not runtime/tool-call50specific, use `product-acceptance-verifier-maintainer` instead. If the user51provides only one failed run id and wants root cause, use52`debug-run-transcript-maintainer` first, then return here for rerun proof after53a fix.5455## Runtime Matrix5657Select the runtime matrix from the user's request and the changed integration.58Test one runtime for a runtime-specific claim. Use the full supported matrix for59an explicit all-runtime claim or a shared adapter/tool change that affects it:6061- Codex62- Claude63- OpenCode64- Pi6566Extend the matrix when the user names additional supported runtimes such as Cursor or67Gemini. Do not treat a Codex pass as proof for another runtime.6869Read only the relevant reference files:7071- `references/probe-workflow.md`: real local setup, disposable data, probe72 script expectations, and proof levels.73- `references/transcript-evidence.md`: how to prove tool use and reject CLI74 fallback from logs.75- `references/codex.md`: Codex MCP transcript shape and known traps.76- `references/claude.md`: Claude MCP transcript shape and strict config traps.77- `references/opencode.md`: OpenCode MCP transcript shape, provider/model78 caveats, JSONL parser expectations, and final-text pitfalls.79- `references/pi.md`: Pi native extension transcript shape, schema/error80 propagation traps, and 429/auth/provider separation.81- `references/reporting.md`: verdict format, mutation ledger, and pass/fail82 language.8384## Acceptance Standard8586A runtime passes only when all are true:87881. The local Rudder source of truth is identified, usually89 `GET /api/health` on `http://127.0.0.1:3100`.902. A real local run was triggered for that runtime, or an existing real local91 run was inspected with enough raw transcript/log evidence.923. The transcript shows the expected Rudder MCP/native tool calls.934. There is no model-visible fallback to shell, Bash, curl, or `rudder` CLI for94 Rudder work.955. Tool results are not internally failed (`isError`, structured error,96 `rudder_cli_command_failed`, missing required argument, auth/org failure).976. The terminal product effect was read back: issue status, comments, run98 status, final text, API result, or another requested surface.997. Provider/model blockers are separated from Rudder adapter/tool blockers.100101If the user requires "all tools", do not only run a happy-path issue workflow.102Run or request a manifest-driven coverage plan: list every exposed tool, verify103schemas load, check runtime-managed identity is not model-provided, and execute104representative read/mutate/file/image/pagination/error cases. Mark unexecuted105tools as not covered.106107## Default Probe Ladder108109Use the smallest probe that answers the question, then escalate only as needed:1101111. **Tool availability probe**: call `rudder_agent_me`, require final answer112 to say MCP/tool path and no fallback.1132. **Issue workflow probe**: seed an issue, then require114 `rudder_issue_context`, `rudder_issue_checkout`,115 `rudder_issue_comment`, and `rudder_issue_done`; read back final issue116 status and comments.1173. **Representative matrix probe**: add read/list/pagination/file/comment/image118 examples such as `rudder_runs_errors`, `rudder_library_file_ref`,119 inbox/context, review, chat, or automation.1204. **Full manifest audit**: prove every exposed Rudder tool has a stable name,121 schema, description, and handler classification; execute safe read-only122 tools and representative mutating tools in disposable data.123124## Known Judgment Rules125126- A successful final answer is not enough. Inspect raw or parsed transcript.127- A tool call name in prompt text is not evidence. Use structured tool-call128 events or parsed run-intelligence entries.129- A run can fail after completing the product effect. Report that separately:130 `tool/product effect passed; runtime finalization failed`.131- A provider 429/auth/model error before any tool call is not an MCP auth/org132 bug. Mark it `blocked_provider`.133- If a tool result contains `isError: true` but the runtime marks the tool call134 completed, treat the runtime as failed or at least suspect until fixed.135- Internal runtime-owned bridges may execute a server process. That is136 acceptable only if the model-visible action is a typed Rudder tool and137 runtime env owns identity. It is not acceptable if the model uses Bash/curl or138 runs `rudder ...` itself for Rudder work.139140## Output141142Use `references/reporting.md` for the full shape. At minimum include:143144- overall verdict: `PASS`, `FAIL`, or `QUESTION` for the requested matrix;145 observed required behavior failure yields `FAIL`; otherwise missing required146 proof or provider access yields `QUESTION`; complete required proof yields `PASS`147- verdict per runtime: `PASS`, `FAIL`, `BLOCKED_PROVIDER`, `PARTIAL`, or148 `NOT_RUN`149- run IDs, issue IDs, org IDs when disposable data was created150- observed tool names151- whether CLI/Bash/curl fallback was observed152- internal tool errors153- terminal product effect154- skipped coverage and why155- mutation ledger156157Do not say "all green" if any runtime is blocked, substituted, partial, or only158covered by simple `agent_me`.