# Test Ocas Openclaw > Regression test the OpenClaw Codex App Server plugin against a live local OpenClaw instance in Telegram or Discord. Use when the user wants an end-to-end OCAS manual test pass, wants to verify binding, `/cas_resume` or `/cas_status` flags for model/fast/yolo, approvals, plan mode, review, compact, status-card controls, split default-vs-full-access permission behavior, stop-button interrupt handling, rename, or other slash-command behavior, or wants reproducible bug notes from a live chat integration run. Use when this capability is needed. - Skill: `tomevault-io/test-ocas-openclaw` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add tomevault-io/test-ocas-openclaw` - Raw SKILL.md: https://api.skillmd.com/api/skills/tomevault-io/test-ocas-openclaw/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: tomevault-io (https://skillmd.com/u/tomevault-io) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/tomevault-io/test-ocas-openclaw --- # Test OCAS OpenClaw Use this skill for manual regression passes of this plugin against a real local OpenClaw instance. ## Preconditions - Confirm the plugin branch under test is linked into the local OpenClaw checkout and OpenClaw is running. - Prefer Playwright MCP or Chrome MCP for Telegram Web. It was more reliable than `agent-browser` in recent passes. - Use [$agent-browser](https://github.com/vercel-labs/agent-browser) only as fallback. Read [references/agent-browser.md](references/agent-browser.md) only if you intentionally choose that path. - If Telegram Web or Discord is not logged in, stop and ask the user to complete login. - Prefer a low-risk Codex thread unless the user asks otherwise. In this repo, `discrawl` and `dupcanon` are safe defaults for resume tests. - Treat `Permissions: toggle` as switching between `Default` and `Full Access`. If Full Access is unavailable in the current Codex Desktop session, expect the status card to stay in Default mode with an explanatory note. ## Canonical State Files Use these when the UI and the plugin disagree: - Core chat binding store: `~/.openclaw/telegram/thread-bindings-default.json` - Plugin state: `~/.openclaw/openclaw-codex-app-server/state.json` Read them when: - a topic looks bound but `/cas_status` disagrees - plan mode appears stuck - a questionnaire should exist but Telegram Web does not render it - slash replies appear in the wrong chat or topic - answer buttons appear but the active pending request does not advance - the status card claims a pending permissions migration that never applies after the run finishes Treat the state files as diagnostic evidence, not as a thing to hand-edit. ## Preferred Browser Path - Keep the browser anchored to the requested topic or channel for the full pass. - For Telegram topics, keep the topic fragment in the URL when possible. - If the UI looks stale but local state shows the expected pending request or callbacks, refresh Telegram Web with `Cmd+R` before calling it a backend bug. - Separate browser-navigation failures from OCAS failures. A browser can drift to `General` or the chat list without proving an OCAS routing bug. ## Regression Flow 1. Establish the target conversation. - Stay in the requested group, DM, channel, or topic for the entire pass. - If the expected Telegram topic is missing, check whether `/cas_resume --sync` renamed it. Rename it back or create a fresh test topic if needed. - If slash-command routing leaks into `General`, log that separately and move the browser back to the intended topic before continuing. 2. Baseline the conversation. - Run `/cas_status`. - If the topic is already bound, run `/cas_detach` first, then confirm the unbound state with `/cas_status`. - For Telegram topics, the reply must appear in the same topic. If it lands in `General`, record a routing bug. 3. Resume a thread. - Prefer `/cas_resume` or `/cas_resume --all` with the normal picker flow. - Avoid `/cas_resume --all ` for now. It is a known regression path and may not surface the expected approval UX. - Cover at least one flag-driven bind path with `/cas_resume --model `, `/cas_resume --fast` or `/cas_resume --no-fast`, and `/cas_resume --yolo` or `/cas_resume --no-yolo`. - If the bind path falls back into project or thread pickers, verify the requested model, fast, and yolo settings still apply after the picker choice and final bind approval. - When approval appears, use Allow Once unless the user asks otherwise. - After binding, send `who are you`. - Expect a Codex response. If the reply sounds like OpenClaw persona text instead of Codex, the bind failed. - Run `/cas_status` again and verify: - `Binding: active` - the expected thread id or thread title - the expected model - `Plugin version: 0.0.0` for local dev branch testing when that is the expected version 4. Verify status-card controls. - Use the interactive `/cas_status` message as the primary control surface. - Expect one status message that edits in place rather than a stream of follow-up bot messages. - Verify the card includes: - `Select Model` - `Reasoning: ...` - `Permissions: toggle` - `Compact` - `Stop` - Verify `Fast: toggle` only appears when the current model supports fast mode. If the current model does not support fast mode, expect no fast button. - Click `Select Model`. - Expect the same message to switch into a model picker. - Choose a different model. - Expect the same message to return to the status card and show the new model on the `Model:` line. - Run `/cas_status` again and confirm the new model is still shown. - Click the reasoning button. - Expect the same message to switch into a reasoning picker. - Choose a different reasoning level. - Expect the same message to return to the status card and show the new reasoning on the `Model:` line. - Run `/cas_status` again and confirm the reasoning state persisted independently of the current model. - If the current model supports fast mode: - click `Fast: toggle` - expect the same message to update `Fast mode:` immediately - run `/cas_status` again and confirm the fast-mode state persisted - If the current model does not support fast mode: - use `Select Model` to switch to a fast-capable model before testing the fast toggle - Click `Permissions: toggle`. - If Full Access is available and no turn is active: - expect the same message to update `Permissions:` between `Default` and `Full Access` - run `/cas_status` again and confirm the new permissions mode persisted - If a turn is active: - expect the same message to stay on the current mode and add a note that the requested permissions mode will apply after the current turn ends - after the run finishes, run `/cas_status` again and confirm the pending profile migration actually applied - If Full Access is unavailable in the current session: - expect the status message to remain on `Permissions: Default` - expect an explanatory note that Full Access is unavailable for the current session 5. Verify approval rendering in Default mode. - Put the conversation back into `Permissions: Default` before this step. - Send this exact prompt as plain text in the bound conversation: ```text I want you to run `npm view dive` and make sure to ask to exit the sandbox as it needs network access. ``` - Expect a real execution approval dialog, not a plain text question about whether the user wants approval. - Expect approval buttons such as `Approve` or the platform-equivalent action controls. - Expect a code-formatted command area that shows `npm view dive`. - Verify the displayed command is trimmed for presentation and does not leak a shell-launcher wrapper like `/bin/zsh -lc ...`. - After verifying the dialog, approve the command with `Approve Once` or the platform-equivalent approval button so later tests are not blocked by a stale pending approval. - If the model only asks a conversational question like `Do you want to allow...` without rendering execution approval controls, treat that test as invalid and rerun with the exact prompt above. 6. Verify full-access execution path when enabled. - Use the status card to switch into `Permissions: Full Access` when it is available in the current session. - Send this exact prompt as plain text: ```text Run `npm view dive version` and reply with only the version string. ``` - Expect the command to run without an approval dialog. - Expect a direct result after a short delay rather than an approval request. - After the command completes, switch the status card back to `Permissions: Default`. - Run `/cas_status` again and confirm the card shows Default mode. 7. Verify long-running review behavior before plan mode. - Run `/cas_review` before `/cas_plan`. - Treat `/cas_review` as a long-running request. It may take several minutes. - Do not interrupt `/cas_review` by starting `/cas_plan`, `/cas_compact`, or other long-running flows while review is active. - After starting review, wait 30 seconds at a time and check for output again. - Record whether review immediately skips expected desktop-style base and branch questions. That is a behavior gap, but not the same thing as a review failure. 8. Verify Stop-button behavior on a long-running run. - Start a long-running Codex action such as `/cas_review`, `/cas_plan`, or a prompt likely to wait on approvals or take more than a few seconds. - While the run is active, press `Stop` on the status card. - Expect the active run to interrupt. - Expect the same status message to refresh in place after the interrupt. - Run `/cas_status` again and confirm there is no active-run note left behind. - If you also want to verify deferred permission migration, start a long-running run, press `Permissions: toggle` during the run, confirm the pending note, then press `Stop` and confirm the requested mode applies only after the run ends. 9. Verify plan mode only after review has finished. - Use `/cas_plan `. Do not test plan mode by sending the raw prompt without the command. - Use a short questionnaire prompt first. A breakfast-choice prompt is good because it exercises question rendering and answer submission. - Expect: - questionnaire text - answer buttons - question progression when answering - final `Implement this plan` controls - If question 1 renders but button clicks or plain-text answers do not advance the active pending request in `state.json`, record that as a real failure. - If plan mode gets stuck, `/cas_plan off` should exit it. 10. Verify long-plan truncation. - Ask for a final plan longer than 4000 characters. - Expect the Telegram preview to truncate and the plugin to attach the full Markdown plan when needed. 11. Cover other control commands as needed. - `/cas_resume` - Verify `--model`, `--fast` or `--no-fast`, and `--yolo` or `--no-yolo` paths on direct bind, picker bind, and `--new` flows. - Expect the chosen settings to show up immediately in `/cas_status` after the bind completes. - `/cas_mcp` - Expect a list of configured MCP servers. - `/cas_skills` - Expect installed skills. - If the response shows both a full text list and buttons for the same skills, record a display bug rather than a blocker. - `/cas_status` - Verify in-place message edits for model, reasoning, fast when supported, permissions, compact, and stop controls. - Verify the card does not post extra bot messages for each button press unless the platform requires it. - Verify `/cas_status --model`, `/cas_status --fast` or `/cas_status --no-fast`, and `/cas_status --yolo` or `/cas_status --no-yolo` update the bound conversation settings and refresh the shown status. - `/cas_compact` - Expect progress keepalives and a final context-usage report. - `/cas_rename` - Verify thread rename and, if requested, `--sync` topic rename behavior. - `/cas_diff`, `/cas_init` - Verify current forward-or-placeholder behavior instead of assuming full implementation. - Treat known-placeholder behavior as neutral, not as a regression, unless the user says the command should already work. - Do not look for `/cas_model`, `/cas_fast`, or `/cas_permissions`. - Those controls were folded into `/cas_resume`, `/cas_status`, and the status card. 12. Clean up. - End by running `/cas_detach`. - Confirm the conversation is no longer bound. - Confirm the topic has no lingering pending request in `state.json` if plan mode, stop, or approvals were exercised. ## Result Legend - `✅` Pass. Behavior matched what should work now. - `❌` Fail. Behavior should work now and did not. - `➖` Neutral. The command is known to be unimplemented, placeholder-only, intentionally incomplete, or not configured in this environment. Use a flat results table while testing: | Area | Status | Observed | Notes | | --- | --- | --- | --- | | `/cas_status` | `✅` | Bound topic reply in-topic | Include thread, model, and plugin version | | `/cas_resume` flags | `✅` or `❌` | `--model`, `--fast` or `--no-fast`, `--yolo` or `--no-yolo` survive bind flow | Include whether a picker path preserved the settings | | Status controls | `✅` or `❌` | Same message edits in place for model, reasoning, fast when supported, permissions, compact, and stop | Record extra messages as a bug | | `/cas_status` flags | `✅` or `❌` | `--model`, `--fast` or `--no-fast`, `--yolo` or `--no-yolo` refresh the binding and status card | Note unsupported fast-model cases separately | | Approval dialog | `✅` | Real execution approval with trimmed `npm view dive` | Approve after verifying | | Full Access execution | `✅`, `❌`, or `➖` | `npm view dive version` runs without approval | Use `➖` if Full Access is unavailable in the current session | | Permission migration | `✅` or `❌` | Default vs Full Access persists across `/cas_status` | Note pending migration if toggled mid-run | | Stop button | `✅` or `❌` | Active run interrupts and status refreshes | Record whether refresh was in-place | | `/cas_review` | `✅` or `❌` | Long-running review | Wait 30s between checks | | `/cas_plan` render | `✅` or `❌` | Question text and buttons | Separate render from answer submission | | `/cas_plan` answer submission | `✅` or `❌` | Active request advances or stays stuck | Compare against `pendingRequests` | | `/cas_diff` | `➖` or `❌` | Placeholder or real command | Use `➖` if still knowingly unimplemented | ## Gotchas - Telegram Web can go stale. If a simple request appears hung for more than about one minute, and the local state file already shows the pending questionnaire or callbacks, refresh Telegram Web with `Cmd+R` before calling it a backend bug. - Plain text can still route correctly even when slash commands from a topic leak to `General`. Record both facts separately. - A core binding can exist in `thread-bindings-default.json` while plugin-local state is stale or missing. Compare both files before concluding which side is wrong. - Telegram can leave stale questionnaire buttons visible in the DOM. If a click produces `No active Codex run is waiting for input.` but `pendingRequests` still shows the active questionnaire unchanged, record that the visible button press did not reach the active request. - Starting `/cas_plan` while `/cas_review` is still running can interrupt the review. Avoid that sequence during normal regression passes. - Permissions migration is edge-triggered on turn completion. If you toggle permissions during an active run, do not expect the profile switch to apply until the run ends or is stopped. - A full-access test only proves the split-profile path when the command runs without approval and later Default-mode commands still trigger approval again. ## Evidence To Capture - Exact command sent - Exact chat or topic where the reply appeared - Whether the reply was plain text, buttons, or an attachment - Whether the status card edited in place or created a new bot message - Which permissions mode was selected before the command under test - Relevant snippets from `state.json` and `thread-bindings-default.json` when UI behavior is suspicious - Known-good versus observed behavior - The result-table row for each command or flow tested ## Formal Bug Notes If the user wants the findings turned into issues, PR notes, or project-board items, use [$project-manager](./.agents/skills/project-manager/SKILL.md). --- > Source: [pwrdrvr/openclaw-codex-app-server](https://github.com/pwrdrvr/openclaw-codex-app-server) — distributed by [TomeVault](https://tomevault.io).