Scenario Quality Gate
Overview
One tool, asset_quality_gate_run, scores a finished image asset and returns a pass/warn/fail
verdict with 0 to 100 scores and per-dimension lists of reasons and suggestions: an AI-quality
check always, plus brand-brief compliance when the team or project has a brief configured. Image
assets only. Quality Gate is an Enterprise add-on: when it is not enabled for the team the call
fails cleanly, so detect that, fall back to an asset_analyze review (see
scenario-asset-analysis), and say so. Retrying never clears it.
The tool is catalog-only and write-class: get the schema with scenario_tools_search, then run it
through scenario_tool_execute_write with {name: "asset_quality_gate_run", parameters: {...}},
scope ids inside parameters (the read executor rejects it by lane and names the right one). Or
reconnect with ?toolsets=full. Connection and scope: see the scenario skill. If a sibling skill
named here is missing from your available skills, load it with skill_get (Scenario MCP) or ask the
user to install it (https://github.com/scenario-labs/skills#install); unattended, proceed from tool
schemas and flag the gap.
Quick reference: what a call costs
| Call |
What happens |
Cost |
| Plain call, usable verdict stored |
Returns the stored verdict, source: "stored" |
Free (the response carries no creativeUnitsCost) |
| Plain call, no usable verdict |
Runs a new analysis and stores the verdict, source: "new_analysis" |
Billed; the response carries creativeUnitsCost (1 CU as of this writing) |
rerun: true |
New analysis that replaces the stored verdict |
Billed |
dry_run: true |
Nothing runs; returns the price of a new analysis as creativeUnitsCost |
Free |
A plain call is a read only when a usable verdict already exists; otherwise the same call silently
escalates to the billed analysis. "No usable verdict" covers both never-scored and a stored error
result. When spend needs approval first, probe free: dry_run: true, or asset_get, whose
qualityGate field carries the verdict and scores when one is stored (summary only; the stored
reasons and suggestions come back through the tool, still free). sensitivity (low, medium,
high; default is the team or project setting) shapes new analyses only and is ignored on stored
reads. rerun: true is for after the brief or the sensitivity changed, nothing else: it re-bills
and overwrites.
Reading the verdict
The response is source plus quality_gate: verdict, overallScore, aiQualityScore,
briefComplianceScore, sensitivity, appliedBriefIds, and details. With no brief configured
the compliance dimension is absent entirely (appliedBriefIds: [], overallScore equals
aiQualityScore); with one, details.briefCompliance sits beside details.aiQuality, each
carrying a score and lists of reasons and suggestions, usually several of each.
Turning the verdict into a better image
A verdict alone only sorts assets. The gate earns its cost when the feedback drives the next
attempt:
reasons name concrete flaws ("elongated finger anatomy on the left hand", "background gradient
off the brief's palette"). Use them to pick the fix path per flaw: a local defect on an otherwise
approved image is a masked inpainting pass (scenario-image); a global one is a regeneration.
suggestions are written as edit instructions, often worded for manual retouching. Apply all that
fit, not just the first: translate them into the next model_run prompt or parameters, or into
the edit instruction.
- A regenerated image is a new asset, so a plain call scores it.
rerun is never part of the loop.
It may even come back pre-scored for free: teams with auto-detect enabled
(qualityGateAutoDetect: true on their teams_list row) score new generations automatically.
- When a round repeats the same flaw classes at an unmoved score, rewording will not fix them:
switch models (
recommend again) or repair the flaw with a masked edit before spending another
round.
- Fix the exit bar and a round cap up front:
verdict: "pass" by default, or a score target the
user names (overallScore at or above 90, say; the named bar then outranks a bare pass), and
three rounds unless told otherwise, since every round bills a generation plus an analysis. At the
cap, or when a round stops moving the scores, stop: report the best asset with its remaining flaws
and ask before spending more rounds; unattended, deliver that best asset and flag the miss.
scenario-refine-loop carries the loop discipline beyond the gate: one variable per round,
regeneration from the approved baseline, and fix routing when the criteria go past the brief.
What the verdict does not cover
The gate is an artifact detector. Measured on real assets, three limits to design around:
- Artifacts, not plausibility. A panel whose car carried two different wheels, a whitewall on
one only, and no wheel arch above the rear tire scored 92 and
pass at sensitivity: "high",
with reasons praising the chrome rims that were the defect. What is impossible rather than ugly
needs a structural check of your own: counts and attachment (hands, fingers per hand, limbs,
wheels, each attached to one body), part to whole (a wheel sits inside an arch, a guard sits on
its own blade, a reflection sits under what casts it), and what carries the weight.
- It has never seen your reference.
briefCompliance needs a configured brief, and a brief is
text, so no call can tell you this is not the same car or costume as the plate it was generated
from. Compare the asset against its reference yourself, item by item.
- A composite averages its parts. A 12-panel storyboard scored 88 and
pass while one of its
panels was physically impossible. Score the unit you are willing to reject: split a composite with
model_scenario-image-slicer, score each part, repair only the failures, and recompose with
model_scenario-compose-image.
The score is also a weak signal that a structural repair landed: a broken panel and its corrected
replacement both scored 92, and only the reasons changed. Re-read the flaw list, do not watch the
number.
Worked example: iterate a hero prop to pass
- Generate per
scenario-image: model_run, jobs_wait, collect the asset id.
scenario_tools_search with query="quality gate" once for the schema, then dry_run: true on
the first asset to surface the per-analysis price.
- Score:
scenario_tool_execute_write with
{name: "asset_quality_gate_run", parameters: {asset_id, team_id, project_id}}. It returns
source: "new_analysis" and a quality_gate carrying verdict: "warn",
briefComplianceScore: 58, and details.briefCompliance.suggestions asking for the logo at the
top left and a flatter background.
- Fold both suggestions into the prompt, regenerate, and score the new asset with a plain call (no
rerun).
verdict: "pass": deliver, and file it (collections and tags per scenario-asset-analysis).
Later reads of any scored asset are free stored reads.
- The brief changes next sprint: only then
rerun: true on the assets that must be re-judged.
Common mistakes
- Treating a plain call as a free read: without a usable stored verdict it silently runs and bills
the analysis. Probe with
dry_run or asset_get first when the spend matters.
- Passing
rerun: true out of habit: it re-bills verdicts that were free to read. Its one job is
refreshing after the brief or sensitivity changed.
- Expecting
briefComplianceScore with no brief configured: quality_gate carries it and
details.briefCompliance only when appliedBriefIds is non-empty.
- Applying one suggestion and rescoring each time: the lists usually carry several fixes, and one
regeneration can absorb them all.
- Running it through
scenario_tool_execute_read: write-class, rejected by lane.
- Scoring a video, 3D, or audio asset: image assets only.
- Assuming a fresh upload has no verdict: uploads deduplicate by content, so identical bytes return
the same long-lived asset id whatever the filename, and a re-upload of a file the team scored
before carries its stored verdict.
- Setting
sensitivity on a call that returns a stored verdict: it applies to new analyses only;
stricter scoring of an already-scored asset requires rerun: true.
- Retrying the entitlement failure: Quality Gate is an Enterprise add-on. Degrade to the
asset_analyze review in scenario-asset-analysis and tell the user why.
1---2name: scenario-quality-gate3description: "Use when a generated Scenario image needs a quality check or a brand-brief compliance verdict before it ships: pass/warn/fail scoring, on-brand review against a configured brief, QA gating a batch, pricing or refreshing a stored verdict, or iterating a generation until it clears the gate. Keywords: quality gate, quality check, QA, verdict, brief compliance, on-brand, brand brief, pass fail, score, review, asset_quality_gate_run."4license: MIT5metadata:6 source: https://skillmd.com/skills/scenario-labs/scenario-quality-gate7---89# Scenario Quality Gate1011## Overview1213One tool, `asset_quality_gate_run`, scores a finished image asset and returns a `pass`/`warn`/`fail`14verdict with 0 to 100 scores and per-dimension lists of `reasons` and `suggestions`: an AI-quality15check always, plus brand-brief compliance when the team or project has a brief configured. Image16assets only. Quality Gate is an Enterprise add-on: when it is not enabled for the team the call17fails cleanly, so detect that, fall back to an `asset_analyze` review (see18`scenario-asset-analysis`), and say so. Retrying never clears it.1920The tool is catalog-only and write-class: get the schema with `scenario_tools_search`, then run it21through `scenario_tool_execute_write` with `{name: "asset_quality_gate_run", parameters: {...}}`,22scope ids inside `parameters` (the read executor rejects it by lane and names the right one). Or23reconnect with `?toolsets=full`. Connection and scope: see the `scenario` skill. If a sibling skill24named here is missing from your available skills, load it with `skill_get` (Scenario MCP) or ask the25user to install it (https://github.com/scenario-labs/skills#install); unattended, proceed from tool26schemas and flag the gap.2728## Quick reference: what a call costs2930| Call | What happens | Cost |31| --------------------------------- | ------------------------------------------------------------------------ | -------------------------------------------------------------------------- |32| Plain call, usable verdict stored | Returns the stored verdict, `source: "stored"` | Free (the response carries no `creativeUnitsCost`) |33| Plain call, no usable verdict | Runs a new analysis and stores the verdict, `source: "new_analysis"` | Billed; the response carries `creativeUnitsCost` (1 CU as of this writing) |34| `rerun: true` | New analysis that replaces the stored verdict | Billed |35| `dry_run: true` | Nothing runs; returns the price of a new analysis as `creativeUnitsCost` | Free |3637A plain call is a read only when a usable verdict already exists; otherwise the same call silently38escalates to the billed analysis. "No usable verdict" covers both never-scored and a stored `error`39result. When spend needs approval first, probe free: `dry_run: true`, or `asset_get`, whose40`qualityGate` field carries the verdict and scores when one is stored (summary only; the stored41`reasons` and `suggestions` come back through the tool, still free). `sensitivity` (`low`, `medium`,42`high`; default is the team or project setting) shapes new analyses only and is ignored on stored43reads. `rerun: true` is for after the brief or the sensitivity changed, nothing else: it re-bills44and overwrites.4546## Reading the verdict4748The response is `source` plus `quality_gate`: `verdict`, `overallScore`, `aiQualityScore`,49`briefComplianceScore`, `sensitivity`, `appliedBriefIds`, and `details`. With no brief configured50the compliance dimension is absent entirely (`appliedBriefIds: []`, `overallScore` equals51`aiQualityScore`); with one, `details.briefCompliance` sits beside `details.aiQuality`, each52carrying a `score` and lists of `reasons` and `suggestions`, usually several of each.5354## Turning the verdict into a better image5556A verdict alone only sorts assets. The gate earns its cost when the feedback drives the next57attempt:5859- `reasons` name concrete flaws ("elongated finger anatomy on the left hand", "background gradient60 off the brief's palette"). Use them to pick the fix path per flaw: a local defect on an otherwise61 approved image is a masked inpainting pass (`scenario-image`); a global one is a regeneration.62- `suggestions` are written as edit instructions, often worded for manual retouching. Apply all that63 fit, not just the first: translate them into the next `model_run` prompt or parameters, or into64 the edit instruction.65- A regenerated image is a new asset, so a plain call scores it. `rerun` is never part of the loop.66 It may even come back pre-scored for free: teams with auto-detect enabled67 (`qualityGateAutoDetect: true` on their `teams_list` row) score new generations automatically.68- When a round repeats the same flaw classes at an unmoved score, rewording will not fix them:69 switch models (`recommend` again) or repair the flaw with a masked edit before spending another70 round.71- Fix the exit bar and a round cap up front: `verdict: "pass"` by default, or a score target the72 user names (`overallScore` at or above 90, say; the named bar then outranks a bare `pass`), and73 three rounds unless told otherwise, since every round bills a generation plus an analysis. At the74 cap, or when a round stops moving the scores, stop: report the best asset with its remaining flaws75 and ask before spending more rounds; unattended, deliver that best asset and flag the miss.7677`scenario-refine-loop` carries the loop discipline beyond the gate: one variable per round,78regeneration from the approved baseline, and fix routing when the criteria go past the brief.7980## What the verdict does not cover8182The gate is an artifact detector. Measured on real assets, three limits to design around:8384- **Artifacts, not plausibility.** A panel whose car carried two different wheels, a whitewall on85 one only, and no wheel arch above the rear tire scored 92 and `pass` at `sensitivity: "high"`,86 with `reasons` praising the chrome rims that were the defect. What is impossible rather than ugly87 needs a structural check of your own: counts and attachment (hands, fingers per hand, limbs,88 wheels, each attached to one body), part to whole (a wheel sits inside an arch, a guard sits on89 its own blade, a reflection sits under what casts it), and what carries the weight.90- **It has never seen your reference.** `briefCompliance` needs a configured brief, and a brief is91 text, so no call can tell you this is not the same car or costume as the plate it was generated92 from. Compare the asset against its reference yourself, item by item.93- **A composite averages its parts.** A 12-panel storyboard scored 88 and `pass` while one of its94 panels was physically impossible. Score the unit you are willing to reject: split a composite with95 `model_scenario-image-slicer`, score each part, repair only the failures, and recompose with96 `model_scenario-compose-image`.9798The score is also a weak signal that a structural repair landed: a broken panel and its corrected99replacement both scored 92, and only the `reasons` changed. Re-read the flaw list, do not watch the100number.101102## Worked example: iterate a hero prop to pass1031041. Generate per `scenario-image`: `model_run`, `jobs_wait`, collect the asset id.1052. `scenario_tools_search` with `query="quality gate"` once for the schema, then `dry_run: true` on106 the first asset to surface the per-analysis price.1073. Score: `scenario_tool_execute_write` with108 `{name: "asset_quality_gate_run", parameters: {asset_id, team_id, project_id}}`. It returns109 `source: "new_analysis"` and a `quality_gate` carrying `verdict: "warn"`,110 `briefComplianceScore: 58`, and `details.briefCompliance.suggestions` asking for the logo at the111 top left and a flatter background.1124. Fold both suggestions into the prompt, regenerate, and score the new asset with a plain call (no113 `rerun`).1145. `verdict: "pass"`: deliver, and file it (collections and tags per `scenario-asset-analysis`).115 Later reads of any scored asset are free stored reads.1166. The brief changes next sprint: only then `rerun: true` on the assets that must be re-judged.117118## Common mistakes119120- Treating a plain call as a free read: without a usable stored verdict it silently runs and bills121 the analysis. Probe with `dry_run` or `asset_get` first when the spend matters.122- Passing `rerun: true` out of habit: it re-bills verdicts that were free to read. Its one job is123 refreshing after the brief or sensitivity changed.124- Expecting `briefComplianceScore` with no brief configured: `quality_gate` carries it and125 `details.briefCompliance` only when `appliedBriefIds` is non-empty.126- Applying one suggestion and rescoring each time: the lists usually carry several fixes, and one127 regeneration can absorb them all.128- Running it through `scenario_tool_execute_read`: write-class, rejected by lane.129- Scoring a video, 3D, or audio asset: image assets only.130- Assuming a fresh upload has no verdict: uploads deduplicate by content, so identical bytes return131 the same long-lived asset id whatever the filename, and a re-upload of a file the team scored132 before carries its stored verdict.133- Setting `sensitivity` on a call that returns a stored verdict: it applies to new analyses only;134 stricter scoring of an already-scored asset requires `rerun: true`.135- Retrying the entitlement failure: Quality Gate is an Enterprise add-on. Degrade to the136 `asset_analyze` review in `scenario-asset-analysis` and tell the user why.