Comfyui Workflow

ComfyUI Workflow

Submit workflows, monitor the queue, and retrieve outputs from a local ComfyUI server. This is the workhorse Skill; comfyui-character builds on top of it for character-consistent generation.

Scope

This Skill covers the transport between an agent and a local ComfyUI server. It does not edit workflows, train models, or invent prompts — those concerns live in sibling Skills. It assumes the user already has a workflow JSON they want to run (either a file the user wrote or one of the bundled workflows/*.json examples).

Two equivalent entry points

You can drive ComfyUI from this Skill in either of two ways. Pick whichever the host environment supports best.

A. Python script (CLI-friendly, no MCP required)

# Health probe
python scripts/submit_workflow.py --probe

# Submit a workflow
python scripts/submit_workflow.py \
  --workflow ./workflows/text-to-image.json \
  --prompt "a tabby cat sleeping on a windowsill, morning light, photorealistic" \
  --output-dir ./out

# Check the queue without submitting
python scripts/submit_workflow.py --queue

# Download a specific image
python scripts/submit_workflow.py --download \
  --filename ComfyUI_00001_.png --output-dir ./out

The script lives at skills/comfyui-workflow/scripts/submit_workflow.py (relative to the Plugin root) and accepts COMFYUI_URL from the environment, falling back to http://127.0.0.1:8188. Full flag list in references/api-reference.md.

B. stdio MCP server (no Python required)

When the host agent is MCP-aware, the Plugin's server.mjs exposes three tools that mirror the Python script:

The MCP server has zero npm dependencies. It is a 200-line stdio JSON-RPC server in plain Node. See mcp.json for the registration snippet and server.mjs for the source.

Workflow contract

ComfyUI accepts a JSON object whose top-level keys are node IDs and whose values are {class_type, inputs} records. When the user provides a workflow, do not edit the structure unless they ask. Two patterns are common:

• A workflow file saved from the ComfyUI web UI: keys are numeric strings, class_type matches the node name in the editor, inputs reference other nodes by [nodeId, outputIndex].
• An "API format" workflow produced by ComfyUI's "Save (API Format)" menu: same shape, but with widget values inlined into inputs.

Both work identically when POSTed to /prompt. The Plugin's workflows/text-to-image.json and workflows/image-to-image.json are in the API format.

Overriding prompt values at submit time

The bundled Python script understands a tiny templating convention so you do not have to rewrite the workflow JSON for every run. In a workflow, a CLIPTextEncode node with inputs.text starting with the literal string __PROMPT__ will have that text replaced with the value of --prompt on the command line before submission. This is a no-op for nodes that do not opt in.

{
  "30": {
    "class_type": "CLIPTextEncode",
    "inputs": { "text": "__PROMPT__", "clip": ["4", 0] }
  }
}

For workflows without that marker, the user must edit the workflow JSON directly.

Step-by-step recipe

1. Health probe. Either call check_queue via MCP or run submit_workflow.py --probe. Stop and report if ComfyUI is unreachable; do not invent a successful run.
2. Resolve the workflow. If the user named a file, read it. If they named a workflow by description (e.g. "the one with the upscale at the end"), ask for a path or a saved name.
3. Apply prompt override if used. If --prompt is provided and the workflow contains __PROMPT__, substitute. Otherwise pass the workflow through unchanged.
4. Submit. POST to /prompt with {"prompt": <workflow>}. Capture prompt_id.
5. Poll. Loop GET /history/<prompt_id> with a 2-second sleep. Stop on terminal status (success, error, or cancelled). Use a 15-minute cap; long jobs should be split.
6. Resolve outputs. The outputs object lists the SaveImage / VHS_VideoCombine files produced. For each, fetch GET /view?filename=...&subfolder=...&type=output.
7. Report. Tell the user the prompt_id, the time taken, the saved file paths, and any warnings (low VRAM, retries, partial outputs).

Long jobs and VRAM safety

• ComfyUI 22B-class models can hold the GPU for hours. The Plugin does not implement a watchdog; for jobs longer than 30 minutes, instruct the user to watch nvidia-smi themselves.
• If a poll returns an error status, capture the full outputs block and the error message; report both. Do not retry automatically — generation is expensive.

Requirements

• ComfyUI running locally (or reachable at COMFYUI_URL).
• Python 3.10+ if using the script. The MCP server requires Node 18+.
• No additional accounts, paid services, or network destinations.

License

Apache-2.0. See LICENSE.