Tosca Automation

Tosca Automation

Use this skill for any task involving the Tricentis TOSCA Cloud REST API or the bundled tosca_cli.py:

• Test cases — create, update, clone, patch work state, export/import TSU
• Modules — create or update Html (web) or SAP GUI modules with locator attributes
• Reusable blocks — extend parameters, wire block references into test cases
• Inventory — search, move, organize into folders
• Playlists — list, run, check results
• Web automation (Html engine) — use Playwright to discover element locators, build modules, assemble 4-folder test cases
• SAP GUI automation (SapEngine) — create screen modules with RelativeId locators, wire the Precondition block
• Any TOSCA Cloud REST API operation not listed above

Two-phase approach — explore first, then build

Authoring a test case is always two stages. Mixing them is the most common source of bad locators: agents that build from a remembered DOM shape rather than a freshly observed one produce modules that match multiple elements, miss below-the-fold content, or pin stale CSS classes.

Stage 1 — Explore the target interface. Use an interface-matched inspector to observe the actual element, confirm uniqueness, and read back ground-truth values (text, href, class, viewport position, RelativeId, etc.). This stage is outside TOSCA — the tool depends on the target:

Stage 2 — Build, run, debug with this skill. Everything TOSCA-side (module / case / block / playlist authoring, inventory moves, TSU import/export, running, reading logs, inspecting failures) goes through tosca_cli.py and ToscaCloudMcpServer per the next section.

The order is non-negotiable: never commit a module whose locator you didn't verify with the exploration tool in the same session. TOSCA accepts ambiguous locators at save time and surfaces the failure only at runtime. A Stage-1 uniqueness check is the only defense.

Transport priority — check in this order on every call

MCP is first. The CLI is the build-and-fallback path. Raw REST is last resort. Secrets stay out of agent context when you go through MCP; an MCP-wired project has already invested in permissions / audit the CLI can't match. Two specific splits apply to TOSCA Cloud though — read them before defaulting to MCP:

1. MCP tools — mcp__ToscaCloudMcpServer__* (RunPlaylist, GetRecentRuns, GetFailedTestSteps, GetRecentPlaylistRunLogs, SearchArtifacts, …) and the Playwright MCP for web-locator discovery. Discover with the host's MCP-listing command (copilot --list-mcp, claude mcp list, Cursor / Windsurf settings). Use the MCP if a matching tool exists, even when the skill's examples show CLI commands.
2. Bundled CLI — scripts/tosca_cli.py. See scripts/README.md for setup. Two cases where the CLI is still mandatory even when MCP is available:
   • Build-time edits (cases create, modules update, blocks add-param, inventory move, cases patch, cases export-tsu / import-tsu) — the MCP server currently exposes a narrower write surface than the CLI. Build with the CLI, run via the MCP.
   • Personal-agent runs still flip the other way: the CLI's service token (Tricentis_Cloud_API) is 403'd on private Local Runner runs. Trigger personal-agent runs via mcp__ToscaCloudMcpServer__RunPlaylist(playlistId, runOnAPersonalAgent=true). See § Iterative test-development loop below.
3. Raw REST — assemble your own HTTP call. Reserve for debugging or when both MCP and the CLI are unavailable.

Agents are allowed (and encouraged) to fix / extend scripts/tosca_cli.py when they hit a bug or missing capability. Rules of the road live in scripts/README.md § Extending this script.

Core principle — always discover before acting

The MBT API has no general list endpoint; use Inventory for user-created artifacts and /packages for built-in modules:

1. inventory search "<name>" --type TestCase — find test case IDs
2. inventory search "<name>" --type Module — find user-created module IDs (built-in Standard modules do not appear here)
3. cases get <id> --json + cases steps <id> --json — ground truth for step composition, module IDs, attribute refs, config params
4. Standard modules (engine-bundled — OpenUrl, CloseBrowser, Wait, Execute JavaScript, HTTP, DB, file, email, T-code, etc.) do NOT appear in inventory search. Discover via GET /_mbt/api/v2/builder/packages. Before building a custom wrapper for any common capability, check there first — see standard-modules.md.
5. Use the discovered JSON as the template when creating or patching similar cases.

Workflow discipline — one artifact at a time

Work sequentially, not in batches. Each build cycle is a complete loop:

1. Discover — inventory search → read an existing similar artifact (cases steps --json / modules get --json) as ground truth.
2. Explore — use Playwright MCP (web) or read similar existing modules (SAP) to confirm element identity before writing JSON. Never commit a module whose locator matches >1 element — verify via browser_evaluate that the count is exactly 1.
3. Build — module → test case → placement, using fresh ULIDs where required (parameterLayerId, businessParameter.id, block-ref parameters[].id). Before finalizing identification choices, action modes, or folder structure, reconcile against best-practices.md (condensed from the 10 official Tricentis Best Practices KBs) — it is the "whether/why" reference that complements the mechanical how-to guides.
4. Run — personal agent via MCP for iterative debug, shared agent via CLI for CI/scheduled runs.
5. Inspect — on failure, read the exact TBox message via GetFailedTestSteps (MCP) or playlists logs (CLI). Classify the failure (see next section) before changing anything.
6. Fix — minimum-diff change: patch the offending module/step, not the whole case.
7. Confirm the write landed — the CLI enforces this automatically. Every write command (cases update / cases patch / modules update / blocks update / inventory patch) issues a follow-up GET and asserts version bumped + target field actually changed, via _confirm_version_bump + _confirm_field → _exit_err → typer.Exit(1) on silent no-op. A green ✓ confirmed means the diff landed; a red Error: … version unchanged means MBT silently dropped the op (unsupported shape: deep JSON pointer paths, remove on array elements, move, or MBT-shape body on Inventory v3). Treat the CLI's exit code as truth. The --no-confirm flag exists to skip this check and must not be used on production writes — it's for debugging transport shape only.
8. Validate — re-run and confirm the step that previously failed now passes. Don't move on until green (or the failure is a documented application defect).
9. Report — IDs (entityId / moduleId / playlistId), folder placement, any remaining gaps.

Don't batch: don't build 5 cases and then run them together. Build one, run it, fix it, then start the next.

No-defect-masking rule

When a run fails, classify BEFORE changing anything:

Forbidden — regardless of reasoning:

• Removing a Verify step to make the run green.
• Changing actionMode: Verify + actionProperty: "Visible"/"InnerText" to a weaker form (dropping actionProperty so the step just interacts).
• Deleting an attribute from a module so a failing lookup stops happening.
• Setting disabled: true on a step that catches a genuine product bug.
• Wrapping a failing Verify in ControlFlowItemV2 If so the test silently skips the bug.
• The re-scoping trap: concluding "this assertion belongs in a different test case" and removing it from the current one. If a step belonged in this case when it was written, it belongs there now.

The only legitimate way to keep a run green while a known product bug exists is to raise the bug and either leave the test failing or set disabled: true with a description linking to the tracker. Masking a defect creates false confidence and defeats the regression suite.

TechnicalId priority (Html engine)

When picking locator parameters for a new Html module attribute, prefer higher-rank options first. Stability beats cleverness — avoid framework-generated class names and long absolute XPaths.

1. Tag + unique Title — stable, locale-independent. Use when the target has a meaningful title="".
2. Tag: INPUT + Name — first-choice locator for form fields.
3. Tag + InnerText — clickable buttons/links with short, unique, stable, locale-appropriate text. Remember InnerText matches the full textContent exactly, including nested children, and is case-sensitive (so it differs from CSS text-transform: uppercase rendering).
4. Tag + HREF + ClassName — nav links. HREF must be absolute; ClassName discriminates between duplicated mobile/desktop/dropdown copies of the same link.
5. Tag + ClassName — last resort. Prefer semantic BEM-style class names; avoid framework-generated hashes like css-xyz123.

Id is silently ignored by the Html engine — never rely on it. After picking a candidate, run a uniqueness check via Playwright MCP:

document.querySelectorAll('<your css>').length   // MUST be 1

If >1, add another discriminator before saving the module. TOSCA will NOT warn you at save time — the ambiguity only surfaces at runtime as Could not find Link '...' or More than one matching ....

Pre-run quality gates

Before triggering a run, confirm these mechanical checks here, and walk through the conformance checklist in best-practices.md (naming, TestCase structure, identification priority, forbidden {CLICK}/{SENDKEYS} patterns, etc.):

• Module has root-level Engine: Html (or SapEngine) configuration parameter.
• Every TestStepFolderReferenceV2 has a fresh ULID parameterLayerId.
• Every parameter value entry has referencedParameterId pointing to a real businessParameter.id on the block.
• version field stripped from PUT bodies (the CLI does this automatically).
• Each attribute locator matches exactly one element on the live page (Playwright MCP uniqueness check).
• Precondition starts with OpenUrl (all 3 params: Url, UseActiveTab=False, ForcePageSwitch=True) and a Wait step for SPAs.
• Leftover-tab handling: on workstation agents that share the user's Chrome, cleanup is wrapped in ControlFlowItemV2 If with a narrow Title="*<AppName>*" — never an unconditional CloseBrowser Title="*".
• Local Runner preflight done (extension enabled in target browser, browser maximized) for personal-agent runs.
• Conformance walkthrough completed — see best-practices.md "Agent checklist" section.

Declarative execution

Act, don't ask. Once the user has approved a task ("build a test for flow X"), execute the full discover → build → place → run → inspect loop without asking for permission between steps. State what you are doing, not what you propose to do.

• ✗ "Shall I create the module first or the test case?"
• ✓ "Creating the module now." (then does it)

Only pause for explicit confirmation on irreversible actions: delete-folder, delete-block, --force, overwriting a test case whose current version you haven't inspected.

Decision tree

Key CLI commands

Invoke the bundled CLI (see scripts/README.md for install + env):

# All examples assume `tosca` is a shim pointing at scripts/tosca_cli.py,
# or invoke directly: python3 <install-path>/skills/tosca-automation/scripts/tosca_cli.py <command>

# Discovery
tosca inventory search "<name>" [--type TestCase|Module|folder] [--include-ancestors]
tosca inventory search "" --type TestCase --folder-id <entityId>
tosca inventory get TestCase <entityId> --include-ancestors

# Test cases
tosca cases get <caseId> --json          # full metadata
tosca cases steps <caseId> --json        # full step tree (use this first)
tosca cases create --name "..." --state Planned
tosca cases update <caseId> --json-file case.json   # full PUT
tosca cases clone <caseId> --name "..."
tosca cases export-tsu --ids "id1,id2" [--module-ids "m1"] [--block-ids "b1"] --output file.tsu
tosca cases import-tsu --file file.tsu

# Modules
tosca modules get <moduleId> [--json]
tosca modules create --name "..." --iface Gui
tosca modules update <moduleId> --json-file body.json

# Reusable blocks
tosca blocks get <blockId>
tosca blocks add-param <blockId> --name <name> [--value-range '1,2,3']
tosca blocks set-value-range <blockId> <paramName> --values '1,2,3,4'
tosca blocks delete <blockId> --force

# Test case patch (partial update)
tosca cases patch <caseId> --operations '[{"op":"replace","path":"/workState","value":"Completed"}]'

# Playlists
tosca playlists list
tosca playlists list-runs
tosca playlists run <id> --wait
tosca playlists results <runId>
tosca playlists logs <runId>                    # per-unit agent logs (E2G, full TBox transcript)
tosca playlists logs <runId> --save ./logs      # save logs.txt + JUnit.xml + TBoxResults.tas + TestSteps.json
tosca playlists attachments <runId>             # SAS URLs per unit (no download)

# Folders
tosca inventory move testCase <entityId> --folder-id <folderEntityId>
tosca inventory create-folder --name "..." [--parent-id "..."]
tosca inventory rename-folder <folderId> --name "..."
tosca inventory delete-folder <folderId> [--delete-children] --force
tosca inventory folder-ancestors <folderId>
tosca inventory folder-tree --folder-ids "<parentFolderId>"

# Simulations (API simulation files)
tosca simulations list [--tags "regression,api"]
tosca simulations get <fileId>
tosca simulations create --name "api-mock.json" --file ./api-mock.json \
    --tags "api,v2" --components "Services,Runnables"
tosca simulations delete <fileId>

# AI helper — natural language → CLI command (OPTIONAL)
# Requires `pip install openai` and `TOSCA_OPENAI_KEY` env var set.
# Don't invoke without first confirming both are in place — otherwise the
# command errors out.
tosca ask "show all failed test cases"          # prints the command
tosca ask "cancel run xyz" --dry-run            # preview only

Install note. Unlike the sibling xray-testing CLI (stdlib-only), tosca_cli.py has four runtime deps: httpx, typer[all], rich, python-dotenv. Declared inline via PEP 723 + also in scripts/requirements.txt. The sdlc-skills installer drops the files but does not pip install them — do that once per project: pip install -r <install-path>/skills/tosca-automation/scripts/requirements.txt. This is a deliberate divergence from xray-testing's zero-dep convention: the upstream CLI is 3.9k lines built on typer + httpx + rich, and a stdlib rewrite would be a multi-week effort with real regression risk. If you'd rather not add the deps, the skill's caveats / references / MCP guidance remain useful; you can call TOSCA Cloud's REST directly.

Enums you'll need when building JSON bodies

• workState: Planned | InWork | Completed
• interfaceType: Gui | NonGui
• Playlist run state: pending | running | canceling | succeeded | failed | canceled | unknown

Critical caveats

ULID generation

The CLI's _generate_ulid() creates Crockford base32 ULIDs. Generate a fresh ULID for:

• Each parameterLayerId in a block reference
• Each businessParameter.id added to a block
• Each parameter entry in a test case's block reference

Step JSON discriminator

Items use $type:

• TestStepFolderV2 — inline folder, children in items[]
• TestStepFolderReferenceV2 — block reference, ID in reusableTestStepBlockId
• TestStepV2 — atomic step
• ControlFlowItemV2 — If/Then conditional

Iterative test-development loop (Local Runner + MCP)

Use this loop when developing a new test case end-to-end on the developer's own machine — fastest feedback because no shared queueing, and the developer can watch the browser drive itself.

One-time prerequisites on the developer machine

1. Install Tosca Local Runner / Cloud Agent — registers a private personal agent under the developer's Okta identity (visible only to MCP, not to the CLI service token).
2. Install + enable the Tricentis Automation Extension in Chrome and/or Edge.
3. Keep the target browser window maximized before each run (minimized → coordinate-out-of-bounds, missed clicks).

The loop

1. Explore the target site with Playwright MCP (browser_navigate → browser_snapshot → identify Tag/InnerText/HREF/ClassName; verify locator uniqueness with browser_evaluate).
2. Build / update modules and the test case via the CLI (service token is fine for build operations).
3. Trigger via MCP — NOT the CLI: mcp__ToscaCloudMcpServer__RunPlaylist(playlistId, runOnAPersonalAgent=true). The CLI's service token is 403'd on personal agents.
4. Wait via MCP: mcp__ToscaCloudMcpServer__GetRecentRuns({nameFilter: "<exact playlist name, including em-dash>"}) — the newly appearing id is the executionId. Poll every 5–10 s, not 60–120 s. Typical single-case runs finish in 15–40 s; a 90 s sleep burns a full prompt-cache window (TTL ≈ 300 s) and can miss the result across 2–5 polls. Never chain sleep N && curl in a retry loop. If two consecutive polls return the same pre-existing set (your run's UUID sorts past the ~10 cap — see caveats), pivot to mcp__ToscaCloudMcpServer__GetRecentPlaylistRunLogs(playlistId) for the authoritative signal.
5. Inspect failures via MCP: mcp__ToscaCloudMcpServer__GetFailedTestSteps({runIds:[<executionId>]}) — returns the per-step failure tree with the engine's exact message + stack trace.
6. Fix the failing module/step/RTSB via the CLI, then back to step 3.

Do not pin AgentIdentifier on the playlist — runOnAPersonalAgent: true is the entire routing instruction, and the playlist stays generic for grid runs too.

When the user says "mcp glitched" / "reloaded mcp" / "screen was locked": do NOT re-dispatch the playlist — the previously triggered run is still executing on the agent, and re-editing test artifacts discards known-good state. Re-issue the last read-side MCP call (GetRecentRuns / GetRecentPlaylistRunLogs) once and continue.

For shared/team-agent runs (CI, scheduled jobs, parameter-overridden runs), use the CLI's playlists run and playlists logs — those work fine under the service-account token.

Preserve the user's flow

When a step fails, fix the step — do not replace the flow with a shortcut. If the user wrote a hover → submenu → click path to reach a page, don't collapse it into a direct OpenUrl to the destination: the test is documenting a user journey, and the shortcut destroys the coverage it exists to provide.

• Only propose a flow change after at least three distinct root-cause fixes have failed, and always ask before applying it.
• Same applies to weakening a Verify step — see § No-defect-masking rule.
• MCP "scaffolding" tools like ScaffoldTestCase are for new test cases, not for copying one. Using ScaffoldTestCase to duplicate a case drops attribute bindings, ControlFlowItemV2 nodes, and parameter values. To clone, use the CLI's cases clone <id> — it fetches the full payload, strips generated item IDs (so the server re-assigns) while preserving module/attr refs, then POSTs as new.

Detailed how-to guides

• Read [Web Automation (Html engine)](references/web-automati

…(truncated)