Langbot Testing

LangBot Testing

Use this skill when an agent needs to verify LangBot behavior through the WebUI instead of only reading code.

Routing

• General WebUI testing: read references/web-ui-testing.md.
• Pipeline Debug Chat: read references/pipeline-debug-chat.md.
• Dify AgentRunner: read references/dify-agent-runner.md.
• Model provider setup or test button: read references/model-provider-testing.md.
• Plugin install/runtime/tool/page smoke: read references/plugin-e2e-smoke.md.
• Local Agent Runner: read references/local-agent-runner.md.
• Local Agent Runner path coverage: read references/local-agent-runner-coverage.md.
• Diff-aware AgentRunner QA after code changes: read references/agent-runner-qa-workflow.md.
• Agent Runner release gate: read references/agent-runner-release-gate.md.
• Sandbox-backed skill authoring: read references/sandbox-skill-authoring.md.
• LangRAG knowledge bases: read references/langrag-knowledge-base.md.
• MCP stdio tool testing: read references/mcp-stdio-testing.md.
• Performance, reliability, or chaos probes: read references/performance-reliability-testing.md.
• Drive a live instance over MCP (not raw HTTP): use the langbot-mcp-ops skill — the instance exposes an MCP server at http://<host>:5300/mcp (reuses API keys). Useful for setting up bots/pipelines/models as test fixtures programmatically.
• Known failures and fixes: read references/troubleshooting.md.
• Reusable test groups: run bin/lbs suite list and bin/lbs suite plan <suite-id> before manually assembling a case set.

Rules

• Read ../.env first and use LANGBOT_FRONTEND_URL and LANGBOT_BACKEND_URL instead of hardcoded ports.
• If a standalone frontend dev server is running, LANGBOT_FRONTEND_URL may point to LANGBOT_DEV_FRONTEND_URL; otherwise it may point to the backend WebUI.
• Confirm the backend and frontend are actually running before testing.
• Run bin/lbs fixture check before fixture-heavy MCP, RAG, multimodal, or plugin smoke tests.
• For runner externalization release checks, run bin/lbs test run agent-runner-release-preflight before the full agent-runner-release-gate suite so configuration blockers are separated from product failures.
• Read Manual Readiness in bin/lbs test plan <case-id>; manual_check means the declared preconditions or setup still need operator confirmation for this run.
• Use an authenticated browser profile prepared by langbot-env-setup.
• Do not expose API keys, OAuth secrets, tokens, or localStorage token values in output.
• A WebUI test is not complete until the visible UI result is checked against backend logs or network behavior.
• A performance result is not complete without metrics evidence and a clear split between LangBot overhead and external provider/tool/network time.
• A chaos or reliability result is not complete until the fault scope, cleanup, and recovery checks are recorded.
• For a suite, use bin/lbs suite start <suite-id> to create the suite evidence root, per-case directories, and suite-start.json/suite-start.md handoff files; use bin/lbs test result <case-id> to write final per-case result.json, then run bin/lbs suite report <suite-id> --evidence-dir <dir>.
• Do not mark a case pass until test result --evidence covers every value in the case's evidence_required.
• For runner-specific Debug Chat cases, use the case-specific pipeline env declared by automation_pipeline_url_env / automation_pipeline_name_env; do not silently reuse a generic LANGBOT_PIPELINE_URL.