AI Oral Coach Skill

AI Oral Coach Skill

This file is the Coding Agent's execution SOP. Any intent involving "build / integrate AI oral coach" — read this file first. All script calls must follow §8 Tool Whitelist and use $SKILL_ROOT absolute paths.

0. Path Baseline (Read First)

Hard rules: ① All script/asset commands use $SKILL_ROOT absolute paths — bare relative paths forbidden. ② Substitute $SKILL_ROOT / $PROJECT_ROOT in templates with real absolute paths before execution. ③ If SKILL_ROOT cannot be found, fallback: find "$PWD" -maxdepth 4 -name SKILL.md -path '*oral-coach*' — never ask the user to move directories.

1. When to Use

Triggers hit when the user message matches §triggers keywords, or contains "TRTC + oral coach / English speaking practice". Not applicable: Customer service / after-sales → use ai-customer-service Skill; non-TRTC ecosystems (Agora etc.) → route to corresponding Skill. Product positioning: voice-first. Text-only requests → advise the user to configure themselves. This Skill does not produce text-only artifacts.

2. Interaction Language & Copy Style

Determine interaction_lang from the message that triggered this Skill (Chinese → zh / otherwise → en). All guidance, questions, and error messages throughout use interaction_lang; switch immediately if the user requests it mid-session. Do not fix the interaction language based on what language this file is written in. Path A UI defaults to following interaction_lang (practice conversations fixed in English); Path B has no UI — only dialogue / comments / docs language.

Copy style (hard rules): All user-facing guidance must be beginner-friendly —

• Use plain language, no jargon; when terms like UserSig / OpenAPI / adapter appear, explain in one sentence what they do;
• Each step says "what we're doing now, what you need to do (usually wait), roughly how long";
• On failure, give "symptom + what to do", never dump raw stack traces;
• Only ask the user to do one thing at a time.

2.5 Prompt Reporting

• Root/host bootstrap handles Prompt reporting and routing. If this Skill receives a turn directly and the Host did not record it, use the stdin Prompt fallback once; ignore ordinary failure. Handle C20 markers per the root dispatcher. On TRTC_REPORTING_NOTICE_REQUIRED_V1, finish the answer first; the installed post-answer Host Hook displays runtime/continuation-notice.md, so do not append or paraphrase it.
• Before a clarification, run the same helper with context --question "<exact question>"; fixed choices still use the standard option UI.

3. Environment Check (Fully Automatic — No User Action Needed)

python3 -c "import sys; assert sys.version_info >= (3,9), sys.version" && echo OK || echo BAD_PY
test -f "$SKILL_ROOT/capabilities/conversation-core/manifest.yaml" && echo OK || echo MISSING
test -f "$SKILL_ROOT/capabilities/conversation-core/.env" && echo HAS_ENV || echo NO_ENV

• BAD_PY → tell user to install Python ≥ 3.9 (https://www.python.org/downloads/). Do not proceed until satisfied.
• MISSING (2nd check) → re-locate SKILL_ROOT using the find fallback from §0 and retry.
• NO_ENV → first cp "$SKILL_ROOT/capabilities/conversation-core/.env.example" "$SKILL_ROOT/capabilities/conversation-core/.env", then proceed to §5 to fill in keys; HAS_ENV → previously configured, can reuse, skip §5 (unless user wants to reconfigure).

Architecture note (for the Agent): All capabilities in this Skill are shipped with the repo. The core server.py uses try_load_capability for pre-wiring — directory present = auto-mounted. Therefore Path A does not need an assembly script — just configure keys and start directly.

4. Path Selection

Use ask_followup_question single-choice:

[{"id":"path","question":"How would you like to start?","options":[
  "A Quick Start — See a full speaking coach right in your browser: conversation, correction, reports. Configure 3 keys, system auto-installs default capabilities, 2-3 minutes to see results",
  "B Integrate into My System — Already have your own app and just need backend capabilities. Get API contracts + integration code, no UI generated; optionally connect your own knowledge base"
],"multiSelect":false}]

A → §6；B → §7. If ask_followup_question is not available, list both paths in natural language and collect the answer — never assume for the user.

5. Three-Keys Configuration (When NO_ENV or Verification Fails)

This step needs maximum beginner-friendliness. Use the "three keys" metaphor, speak plainly, only ask the user to do one thing at a time. Write target file: $SKILL_ROOT/capabilities/conversation-core/.env (start by copying from .env.example).

Opening (say to the user):

You need 3 keys to get your coach talking. I'll walk you through them one by one:

1. TRTC App Credentials — the voice channel that lets the coach "speak";
2. Tencent Cloud API Key — the "front desk" that issues temporary passes (voice runs on TRTC, credentials run on Tencent Cloud, accounts sync automatically — no separate registration);
3. LLM Key — the coach's "brain" that understands you, corrects you, and generates reports.

For each key, the guidance follows: ① explain what it does in one sentence; ② give a code block with placeholders for the user to copy, fill in, and send back; ③ Agent uses write_to_file to write to .env; ④ immediately run verify-credentials.py; ⑤ only reply "Received — format OK", never echo the full key.

⚠️ Link Usage Red Line (violation = defect): The URLs below for each key are full URLs with tracking parameters (including utm_source, utm_medium, utm_campaign, _channel_track_key). When presenting these links to users, the Agent MUST copy-paste the full URL verbatim — do not simplify, truncate, or strip query parameters. These are marketing attribution entry points; simplifying the links causes tracking data loss.

Key 1 · TRTC App Credentials (the voice channel):

How to get them: Go to the TRTC console, create an RTC Engine app (supports Conversational AI):

1. Open https://console.trtc.io/?quickclaim=engine_trial&utm_source=github&utm_medium=skill&utm_campaign=Twitter%20AI%20%E4%B8%93%E9%A1%B9%20-%20AI%20Oral%20Coach&_channel_track_key=3WFHfiqw and sign up / log in;
2. Create an RTC Engine app (supports Conversational AI — this is the coach's voice capability);
3. Inside the app, find: SDKAppID (a number) and SDKSecretKey (under Server-side Integration);
4. ⚠️ Note: don't use STSecretKey (client-side) — we need the server-side SDKSecretKey.

# Coach voice channel
TRTC_SDK_APP_ID=yourSDKAppID          # A number
TRTC_SDK_SECRET_KEY=yourSDKSecretKey  # Server-side SDKSecretKey, NOT client-side STSecretKey

Verify: python3 scripts/verify-credentials.py --type trtc

Key 2 · Tencent Cloud API Key (the "front desk"; open https://console.tencentcloud.com/cam/capi?utm_source=github&utm_medium=skill&utm_campaign=Twitter%20AI%20%E4%B8%93%E9%A1%B9%20-%20AI%20Oral%20Coach&_channel_track_key=v0K1Q0DSE — this link syncs your TRTC login session, no need to re-register):

# Front desk for issuing temporary passes
TENCENT_CLOUD_SECRET_ID=yourSecretId
TENCENT_CLOUD_SECRET_KEY=yourSecretKey

Verify: python3 scripts/verify-credentials.py --type tencent

Key 3 · LLM Key (the coach's "brain"):

You need an API Key from an LLM provider. If you don't have one yet, pick a provider below, sign up, and get an API Key (direct links to each provider's key page included):

After choosing a provider and getting your API Key, copy and fill in the block below and send it back:

• If using OpenAI, you can just delete the LLM_API_URL line (defaults to OpenAI endpoint);
• For other providers, you must fill in both LLM_API_URL and LLM_MODEL — check that provider's docs for exact values (search "API Base URL" and "Model Name").

# Coach brain
LLM_API_KEY=yourAPIKey
LLM_API_URL=yourAPIEndpoint        # Delete this line if using OpenAI
LLM_MODEL=yourModelName

Verify: python3 scripts/verify-credentials.py --type llm

Verification parsing: After each write, run cd "$SKILL_ROOT" && python3 scripts/verify-credentials.py --type <trtc|tencent|llm>. Parse stdout JSON: ok:true → say in plain language "This one's good, on to the next"; ok:false → follow the error / message hint and ask the user to resend that key.

Reports / corrections / suggestions reuse the same LLM_* group by default (leave REPORT_LLM_* empty in .env to fall back); only fill in REPORT_LLM_* if you want a cheaper evaluation model.

5.1 Security Red Lines (Violations Are Defects)

• Keys are only written to .env via write_to_file, never passed as CLI arguments to scripts;
• Never echo full keys in conversation (only confirm "Received — format OK"); no cat .env / echo $SECRET;
• chmod 600 the .env file after writing.

6. Path A: Quick Start

Default artifact: 3-screen SPA (Setup → Practice → Report), replicating the ai-oral-coach/ Demo. Capabilities shipped with repo + pre-wired — no assembly script needed. Agent drives the entire process, user only sees results.

First tell the user what will be automatically set up: scenario practice / sentence correction / reply suggestions / 4-dimension report (all default adapters + built-in question bank).

Built-in UX features (shipped in the SPA — the Agent should preserve these when recreating):

• Reply pace switch (Setup → More settings): Auto reply = full-duplex (server auto-detects end of speech via TurnDetectionMode=0 + user can interrupt via InterruptMode=0; for advanced learners); Think first = push-to-talk tap-to-speak / tap-when-done flow using TurnDetectionMode=1+InterruptMode=1 and manual InvokeLLM (gives beginners time to think). Default Auto reply.
• Conversation mode switch: Coach = after each turn with a mistake, a dismissible correction flashcard pops up (text only, no voice, does not interrupt the conversation); Buddy = only the inline ✱ marker next to the bubble. Default Coach.
• Dark mode: top-nav moon/sun toggle, persisted in localStorage (coach-theme), applied before paint to avoid FOUC; first visit follows OS prefers-color-scheme.
• In practice, the mic button label reflects state: Listening/Tap to speak (Auto pace) or Tap to talk/I'm done (Think-first pace).
• All UI copy is localized in 5 languages (zh-CN / zh-TW / en / ja / ko) in i18n.js. Language is UI-only — the speaking practice is always English (STT fixed to en, English TTS voices). New UI strings must be added to all 5 language tables.

UI design standards: no emoji, colors via CSS variables, 4px spacing grid, font Inter/SF Pro. See $SKILL_ROOT/references/design-specs.md.

Windows note (Agent): start.sh is bash-only. On Windows PowerShell, do the equivalent natively: python -m venv .venv; install with .venv\Scripts\python.exe -m pip install -r capabilities\conversation-core\requirements.txt; launch with $env:WEB_DEMO_DIR="<...>\ai-oral-coach-demo"; $env:PYTHONIOENCODING="utf-8"; .venv\Scripts\python.exe -m uvicorn src.server:app --host 0.0.0.0 --port 8000 (run from capabilities\conversation-core). Set PYTHONIOENCODING=utf-8 — the console is cp1252 and crashes on the scripts' Chinese output otherwise.

All set! Open in your browser:
  · Oral Coach         http://localhost:8000
  · API Docs (Swagger) http://localhost:8000/docs
  · Health Check       http://localhost:8000/api/v1/health
Try it: pick a scenario (optionally set reply pace / conversation mode / dark mode in More settings) → Start → speak English → see live subtitles + correction flashcards → End → view 4-dimension report.

6.1 After Launch: Output Advanced Config Hints (Passive Mode)

Key rule: After Path A is running, only output the plain text hint below — never proactively trigger ask_followup_question. Wait for the user to express the corresponding intent before following §6.2 / §6.3 / §6.4 guidance. This avoids interrupting users who just finished setup and doesn't force them into configurations they don't need.

After Path A succeeds, output this fixed text (verbatim, interaction_lang=en version; adapt for zh):

Demo is up and running! It uses the built-in question bank and your LLM — you can start practicing right away.

If you want advanced configuration later, just tell me — no need to decide now:

  1. Connect your own knowledge base
     Replace Setup screen content with your own teaching materials (supports Dify / Coze / custom REST API).
     → Tell me "I want to connect my knowledge base"

  2. Switch STT / LLM / TTS models
     Change models and parameters — including checking which combinations TRTC supports.
     → Tell me "I want to switch model config"

  3. Customize the frontend/backend for deeper integration
     If you've connected an external KB or switched models and want the Setup screen UI to follow,
     you'll need to adapt the frontend and backend — I'll give you detailed guidance then.
     → Tell me "I want to customize the frontend/backend"

6.2 Connect Your Own Knowledge Base (Only After User Triggers)

Trigger intent: User says "connect my knowledge base" / "use my own teaching materials" / "connect Dify" / "connect Coze", etc.

After triggered, use ask_followup_question to confirm the type:

[{"id":"kb","question":"Which type of knowledge base are you using?","options":[
  "Dify",
  "Coze",
  "My own REST search API"
],"multiSelect":false}]

After selection, guide per the table below. All are .env KB_* variable additions — no code changes needed:

After configuring, unified wrap-up steps for all three types:

1. Restart the service: cd "$SKILL_ROOT" && WEB_DEMO_DIR="$PROJECT_ROOT/ai-oral-coach-demo" nohup bash start.sh > /tmp/coach-start.log 2>&1 &
2. Verify retrieval works: curl -sS -X POST http://localhost:8000/api/v1/kb/retrieve -H 'Content-Type: application/json' -d '{"query":"travel scenario, beginner level English speaking practice topic","top_k":3}' — expect non-empty records
3. Go back to the browser, pick a scenario category, and check that the Setup screen topics / roles now reflect your knowledge base content
4. Fallback note: If retrieval fails or returns empty, the Setup screen silently degrades back to the built-in question bank — it won't error or freeze. If you don't see changes, try step 2 first before suspecting a code issue
5. If the user also wants the Setup screen UI (role cards, recommended expressions, etc.) to dynamically reflect knowledge base content, point them to §6.4 frontend/backend customization guidance

6.3 Switch STT / LLM / TTS Models (Only After User Triggers)

Trigger intent: User says "switch model" / "change STT" / "change TTS" / "use a different LLM" / "what voice models does TRTC support" / "adjust speech rate", etc.

After triggered: Attempt to load the co-repo trtc-docs Skill to answer. The path is $SKILL_ROOT/../trtc-docs (relative to the repo root containing this Skill). Check existence with test -f "$SKILL_ROOT/../trtc-docs/SKILL.md".

Agent execution rule:

• If $SKILL_ROOT/../trtc-docs/SKILL.md exists → read it and follow its SOP to answer the user's STT/LLM/TTS questions.
• If not → directly consult the TRTC Conversational AI official docs below and answer based on them. Do not mention any internal paths or repo information.

Documentation entry points (specific values subject to docs):

If the user's question involves specific model provider params (e.g., OpenAI, DeepSeek, Minimax, etc.), further consult the provider-specific sub-documentation based on the above overview page.

Config reference (for the Agent to proactively inform the user; specific values subject to docs):

Security: All parameter values must only be written to .env — no CLI plaintext pass-through (see §5.1).

6.4 Frontend/Backend Customization Guide (Only After User Triggers)

Trigger intent: User says "customize frontend/backend" / "Setup screen follows KB" / "make UI dynamic" / "how to modify frontend", etc.

After triggered, inform the user of the following key points (don't modify code directly unless the user explicitly asks):

Background (clarify first to avoid unrealistic expectations)

The current Demo Setup screen has two layers of data:

• Top-level scenario cards (Travel/Work/Daily/Free — title, description, role display): hardcoded in the frontend i18n.js, not affected by external KB. This layer is a brand/product fixed frame — usually doesn't need changes.
• Specific scenario candidates (AI opening, topics, role-playing): already behind the backend GetSceneCandidates endpoint. With KB configured, the backend automatically uses KB content, but it's currently fetched only when the user clicks "Start" — the Setup screen does not display specific candidate content — this is the root cause of "not seeing changes."

If you just want openings/topics to follow the KB (minimal change)

No frontend/backend changes needed — the KB is already active: after entering the conversation, the AI's first sentence (Opening) is already from KB content. Let the user confirm this first, then decide whether they need Setup screen pre-display.

If you want the Setup screen to show KB topics and roles in advance (advanced)

Frontend changes needed, backend is nearly unchanged. Change directions and pitfalls:

1. Move query earlier: Pull GetSceneCandidates(scenario, level) from "click Start" to "user switches scenario/level on Setup screen". Async fetch and cache to state.pickedCandidate.
2. WYSIWYG: Setup screen displayed candidate must match the actual candidate used when entering conversation — on "Start" click, directly reuse state.pickedCandidate, don't re-randomize, otherwise "previewed Role A, actually entered Role B".
3. Loading state fallback: KB has network latency (max 8s timeout). On scenario switch, render with local static role names first; fade-replace with KB results when they arrive — no blank screens or stuttering.
4. Silent degradation on failure: KB returns empty or fails → frontend directly keeps local fallback, no error — the backend scenario_source.py already handles degradation, frontend only needs nil-checks.
5. Recommended expressions (optional): If you want KB content to also provide recommended expressions, add an optional Expressions field to the contract format in scenario_source.py, and have the frontend prioritize KB-sourced values when rendering recommendedExpressions.

If you want the top 4 category cards to also be KB-driven (not recommended)

This path has a large blast radius (backend SCENARIOS enum, env.yaml ScenarioPrompts, SystemPrompt assembly all need refactoring), and Dify/Coze APIs have no "list all categories" semantic support — cost far outweighs benefit. Unless there's a clear product requirement, do not go this route.

Verification suggestions after customization

1. Switch scenario/level → wait 1-2s → check if Setup screen detail area role name/topic changes
2. Click Start → AI's first sentence (Opening) matches Setup screen displayed content
3. Disable KB (KB_ADAPTER=none) → Setup screen reverts to built-in bank → compare differences between the two cases

Path A Don'ts: Bare relative paths to call scripts; pass keys via CLI; modify capabilities/*/src/core/; git commit (unless user requests).

7. Path B: Integrate into My System (Backend Capabilities Only, No UI)

Positioning: Integrate oral coach backend capabilities into the user's existing project (PROJECT_ROOT). No UI generated, but full API integration code is generated — so users can immediately call all coach APIs from their own frontend/backend.

1. Three Keys: Same as §5 (voice core hard-depends on all three) → verify-credentials.py --type all.
2. Capability Selection (ask_followup_question multi-select): scenario-roleplay / quick-correct / reply-suggestion / ability-report / custom-learning-kb (connect your own knowledge base). Capabilities are shipped with the repo; selection determines "which contracts to deliver + whether KB is configured".
3. Start Core for Verification: cd "$SKILL_ROOT" && WEB_DEMO_DIR="" nohup bash start.sh > /tmp/coach-start.log 2>&1 & (no WEB_DEMO_DIR → no UI mounted). Verify: /api/v1/health status:ok + POST /api/v1/agent/start returns TaskId.
4. Deliver API Contract + Integration Code:
   • Contract only: cd "$SKILL_ROOT" && python3 scripts/add-capability.py --target-project "$PROJECT_ROOT" (writes INTEGRATION_SPEAKING_COACH.md)
   • Contract + Integration Code (recommended): cd "$SKILL_ROOT" && python3 scripts/add-capability.py --target-project "$PROJECT_ROOT" --apply --tech-stack <web|python|auto>
     • auto: auto-detect project tech stack (package.json → web, requirements.txt → python)
     • web: generates oral-coach-client.js — headless JS client wrapping all APIs + TRTC callbacks
     • python: generates coach_client.py — Python client wrapper with full API coverage
     • Generated code contains no UI components — only API calls + TRTC SDK wiring; UI is rendered by the user
     • If tech stack cannot be matched → fallback to generic-integration.md (L3 generic guide)
5. custom-learning-kb (do only if selected): Configure KB_ADAPTER=dify|coze|user_custom + corresponding KB_* in .env; verify POST /api/v1/kb/retrieve. Note: KB is the only outbound — only interface specs + sample code are provided (auto_adapters/integration_templates/custom-learning-kb-sample.md), no adapter code generated (UserCustomKBClient is already built into clients.py).

7.1 API Contract Types (Delivered to the User Developer)

• inbound (we expose, your frontend calls): /api/v1/config /agent/start|stop|farewell|invoke /scene/generate /correct /suggest /report.
• outbound (only custom-learning-kb, we fetch from you): /api/v1/kb/retrieve → Dify/Coze/your REST.

  The oral coach is almost entirely inbound (self-contained coach API); this is the biggest difference from AI customer service — which is mostly outbound (connecting to ticketing / KB).

7.2 Swap the Brain (Core Selling Point)

Coach capability "adaptation" = swap evaluator / prompt, never touch core: change LLM_* / REPORT_LLM_* in .env, override prompts in each capability's src/adapters/default.py, or add src/adapters/user_custom.py and set <XX>_ADAPTER=user_custom. See references/evaluator-port.md for details.

Path B Don'ts: Generate any UI (do not set WEB_DEMO_DIR); bare relative paths; replace mocks with real systems on behalf of the user (only give specs + adapters); modify */src/core/.

8. Tool Whitelist

Forbidden: Plaintext keys as CLI arguments; cat .env / echo $SECRET; git commit/push (unless user requests); bare relative paths to call scripts. write_to_file limited to: .env (keys), /tmp/*, integration files in user's project, user_custom.py; other file writes require user confirmation. Never hand-edit */src/core/.

9. Common Issues

Final Reminders for the Agent: ① §0 path baseline first — always absolute paths, never ask user to move directories. ② At each step, call tools to get facts first, then explain to user — don't answer from memory. ③ Tool failures → give stderr summary to user, don't hide errors. ④ Path A needs no assembly script (capabilities pre-wired) — configure keys → deploy UI → start.sh → health check → give entry → §6.1 advanced hints. ⑤ Path B never generates UI (no WEB_DEMO_DIR) — verify core e2e + deliver inbound contract + --apply to generate integration code (no UI involved) + configure custom-learning-kb as needed. ⑥ custom-learning-kb works in both Path A and Path B; not connecting it does not affect any default experience. ⑦ Strictly follow §5.1 security red lines and §8 whitelist. ⑧ The Root/Host reporting boundary handles the Prompt before each user turn; do not add per-action reporting commands.