Comfyui

ComfyUI Workflows in Video Production Buddy

Use this skill before calling comfyui_image or comfyui_video, and when converting a community ComfyUI workflow into a Video Production Buddy tool call.

Server Contract

• ComfyUI must be running before the tool can generate. The default server is http://localhost:8188; override it with COMFYUI_SERVER_URL.
• Health and hardware status come from GET /system_stats.
• Jobs are submitted to POST /prompt, completed outputs are read from GET /history/{prompt_id}, and artifact bytes are downloaded with GET /view.
• Export workflows with ComfyUI's API-format JSON, not the UI layout format. If a downloaded workflow will not submit, re-export it from ComfyUI with API format enabled.

Choosing a Workflow

• Use bundled workflows when the requested operation matches and the local machine has the required models and VRAM.
• Use a custom workflow_json or workflow_path when the user needs a community recipe, a lower-VRAM model, a different style family, or custom nodes.
• For 8GB-12GB GPUs, prefer lower-footprint workflows such as Wan 2.1 1.3B, LTXV FP8 or quantized workflows, or Wan 2.2 GGUF/quantized community workflows. The bundled Wan 2.2 14B FP8 video workflows are a 16GB-class path, not a provider-wide floor.
• Do not promise that arbitrary custom workflows will fit a machine. The workflow, quantization, resolution, frame count, and offload settings determine the real resource envelope.

Output Node Contract

• Custom workflows must pass output_node.
• Pick the node that writes the artifact, usually SaveImage, SaveVideo, VHS_VideoCombine, or another terminal saver node.
• Pass the node ID as a string, for example "108". Do not pass the class name.
• If a workflow has multiple savers, choose the final deliverable node, not previews or intermediates.

Templated vs Fixed Nodes

• Identify templated nodes before execution: prompt text, seed, dimensions, frame count, source image, sampler settings, and output filename prefix.
• Fixed nodes are model loaders, VAEs, text encoders, LoRA loaders, schedulers, and graph wiring. Do not mutate those unless the workflow author intended that customization.
• For community workflows, inspect each loader node and note every required model or custom node before running. Missing models should be handled through the tool's structured missing_models payload when available.

Model and LoRA Setup

• Use ComfyUI Manager or the workflow author's model links when available, and respect model licenses.
• Place models in the folders expected by the loader nodes: diffusion models under ComfyUI/models/diffusion_models/, text encoders under ComfyUI/models/text_encoders/, VAEs under ComfyUI/models/vae/, and LoRAs under ComfyUI/models/loras/.
• For LoRA stacks, use LoraLoader or LoraLoaderModelOnly chains in the workflow. Record each LoRA name plus strength_model and strength_clip when applicable.
• The current ComfyUI tools do not inject LoRAs into arbitrary graphs. To use LoRAs, provide a workflow that already contains the LoRA loader chain and pass model-stack provenance.

Provenance

• For custom workflows, provide workflow_name and workflow_model when known.
• Provide workflow_model_stack for reproducibility when the workflow is not bundled. Include base checkpoint or diffusion model, quantization, text encoder, VAE, LoRAs and strengths, sampler or scheduler, steps, and guidance if the workflow exposes them.
• The tools record the final workflow hash. Treat that hash plus the model stack, seed, dimensions, and prompt as the reproducibility contract.

Failure Handling

• If the server is unavailable, surface the structured setup offer from the provider entry. Starting ComfyUI or setting COMFYUI_SERVER_URL is the first fix.
• If models are missing, read data.missing_models[]; each item should include the file name, role, destination hint, and download URL when Video Production Buddy knows it.
• If custom nodes are missing, ask the user to install them through ComfyUI Manager or the workflow author's documented install path, then restart ComfyUI.
• If a long render times out locally, check ComfyUI history before retrying from scratch; the server may still have completed the prompt.