Comfyui

ComfyUI: driving the local install

Use this whenever the task involves generating or rendering images, video, or audio with ComfyUI, or building/running a ComfyUI workflow. Read it first, then act.

Files in this kit (pull the right one on demand)

Only this SKILL.md auto-loads; everything else is read when relevant, so route to it instead of leaving it unread. Two layouts, same files: in the INSTALLED skill everything sits flat next to this file, so docs/TASKS.md below means TASKS.md here and docs/NODE_LIBRARY/ocio.md means NODE_LIBRARY/ocio.md; likewise a script the docs name under shared/tools/ in the repo sits in tools/ next to this file once installed. In the repo those prefixes are literal. If a path does not resolve, drop the docs/ and look next to this file before concluding the file is missing.

• MODELS.md (next to this file) - the INDEX of per-model prompt recipes. Look the model up in its table, then read that family file under MODELS/ BEFORE writing the prompt. Two reads, not one: the index does not carry the recipes.
• the sibling minimax-h3 skill (invoke it by name; on disk it sits beside this skill, ../minimax-h3/ on Claude Code and Codex, minimax-h3/ on Gemini and Qwen) - the dedicated MiniMax H3 (Hailuo 3) skill: prompt format (the three named fields, <d> dialogue, camera vocabulary), reference labelling, quants and acceleration, and a symptom-to-cause table. Read it for ANY H3 prompt or local-weights question; MODELS.md keeps the node-level detail.
• the sibling krea skill (invoke it by name; beside this skill on disk) - the dedicated Krea skill: the fork between Krea's hosted API (Krea2ImageNode / Krea2StyleReferenceNode, per-image pricing, moodboards, capped at 1K) and its open weights, the FLUX.1 Krea Dev graph, Krea Realtime 14B and why its only ComfyUI pack is an unproven lead, and the Krea 2 custom-node packs for ControlNet / identity editing / conditioning control. Read it for the API path or the model choice; MODELS.md keeps the local Krea 2 graph.
• the sibling seedance skill (invoke it by name; beside this skill on disk) - the dedicated ByteDance Seedance skill: the three task types and the word that switches between them, the @Image 1 label syntax, the full-width symbol set, shot sequencing, the asset-count rule, the timing rules that reversed in 2.5, and a failure table. Read it for ANY Seedance prompt; MODELS.md keeps the node-level detail and the price maths.
• docs/TASKS.md - a named common job (generate image / video / audio / 3D, upscale, remove background): the local end-to-end flow for that task, a shortcut layer over this manual.
• docs/NODE_LIBRARY/smart-upscaler.md - our Smart Upscaler pack (11 nodes): tiled upscaling that writes a separate verified prompt per tile. Read it when a tiled upscale of a BUSY or MIXED scene keeps producing confidently wrong tiles or disagreeing seams; the cheaper sampler-tilers in ADVANCED.md stay the right call for uniform subjects.
• docs/MODEL_INDEX.md - the full classified list of all 160 models (recipe / utility / template-only); check whether a named model has a recipe, is a utility, or is template-only.
• docs/NODE_LIBRARY/training.md - the nodes that let a graph MAKE a model, not just prompt one: core's TrainLoraNode / SaveLoRA / LossGraphNode plus the 16 dataset nodes (MakeTrainingDataset, ResolutionBucket, the image-text loaders, video temporal crops) and the full chain wired end to end. Read it whenever someone asks to train or fine-tune anything; docs/TASKS.md has the short route.
• docs/ADVANCED.md - hard tasks: real strengths, gotchas + workarounds, temporal stability, high-detail matting, crop-and-stitch inpaint, PBR, and the verified tool table with licenses.
• docs/KNOWN_ISSUES.md - read BEFORE building, so you do not wire around a currently-broken path.
• docs/NODE_LIBRARY/_INDEX.md - the per-node reference (Nuke-style): for any node, what each input / output is for, how it behaves, bugs + fixes, anti-patterns, and where it slots in a graph. Start here for ANY node question, then query get_node_info for live I/O. When you use or meet a node not in it, add the entry before finishing (docs/NODE_LIBRARY/_SCHEMA.md).
• workflow_layout.py - before saving ANY workflow you build, arrange and verify it IN CODE: auto_layout(wf) positions nodes left-to-right by dependency depth with parallel branches stacked and ZERO overlaps; inspect(wf) reports overlaps / crossings / bounds from the coordinates; fit_group(wf, title) wraps the laid-out nodes in a backdrop that FULLY covers the functional group (edge to edge, none sticking out). NEVER judge a graph's layout from a screenshot (it burns tokens, and clients hit the same wall) - read the positions.
• docs/NODE_LIBRARY/ocio.md - our own ComfyUI-OCIO pack, v1.3.0 (eleven Nuke-style OpenColorIO nodes: Read / Write / Player, six color operators, and the OCIO VAE Decode / VAE Encode pair that decodes without the stock 0..1 clamp; published, github.com/SlavaSexton/ComfyUI-OCIO). Read it for ANY color-management / VFX color task (load a sequence, grade in ACES, write ProRes / EXR, keep values above 1.0 and below 0 out of a generative model). v1.3.0 renamed OCIO Write's from_colorspace to input_colorspace, and an API-format graph carrying the old key is refused, so read the file before building one.
• docs/BUILDING_NODES.md - the hard-won field guide to WRITING a custom node pack (widget order, the combo-validation trap, the JS front-end, server routes, ComfyUI facts, verify-on-real-files). Read it first when you write or modify a custom node, alongside the comfyui-node-* skills.
• docs/KIJAI.md - the kijai ecosystem (his ComfyUI wrappers and nodes: Wan / Hunyuan / CogVideoX / Florence2 / KJNodes / SUPIR / FramePack / SAM2 / FluxTrainer / IC-Light / DepthAnythingV2 and ~50 more) - what each does + node I/O, what is active vs legacy by date, and the supersede map (old -> better). Read it for ANY kijai tool, and to pick the current option over a sunset wrapper.
• docs/NODE_LIBRARY/radiance.md - reverse-engineered reference for fxtdstudios/radiance (a pro 32-bit color-science / HDR / VFX suite, 78 nodes + a v3 rewrite; the strongest public pack in our OCIO / color domain). Read it for a color-science / HDR / VFX-viewer reference implementation, when improving OUR ComfyUI-OCIO pack (it carries the ranked "what to steal" list: processor caching, WebGL 32-bit viewer, OpenEXR writer, LogC3-EI / LogC4), or when a task touches radiance nodes. Its node-building lessons are folded into BUILDING_NODES.md.
• docs/LTX2_TRAINING.md - when the user works with LTX-2 and wants behavior a LoRA captures, offer to train one (official Lightricks trainer).
• docs/EXAMPLE_WORKFLOWS.md - worked end-to-end examples + the multi-model image-edit shootout.
• docs/NODES.md the in-graph Claude nodes (billing / purpose); docs/LAYERS.md the four install layers; docs/BOOTSTRAP.md first-run machine setup; docs/AGENTS.md per-agent matrix (Claude / Codex / Gemini / Qwen); docs/UPDATING.md the weekly model + bug update loop.

Your machine

Read machine.md next to this file before the first ComfyUI call of a session. It carries this install's real paths, GPUs, template-library location and launch command. It is deliberately NOT part of this file: the installer overwrites SKILL.md on every update, so a machine block living here was destroyed by the next git pull plus reinstall, and the bootstrap had to be redone (2026-08-06 audit). machine.md is written once and never overwritten.

If machine.md is missing or still full of angle brackets, run the bootstrap now (docs/BOOTSTRAP.md): health_check, or comfy_client.alive() plus GET /system_stats and /object_info, then write the real values into machine.md. Never assume another machine matches an example.

The four layers (what this kit installs)

1. Knowledge + client: this SKILL.md and comfy_client.py (stdlib, no deps).
2. MCP driver: comfyui-mcp (artokun, MIT): ~90 structured tools so Claude operates ComfyUI directly (generate, build/edit/validate graphs, model download, queue, VRAM, diagnostics, restart). Prefer its tools over hand-POSTing /prompt when present. It rides the MCP SDK's 1.x line, i.e. the revision BEFORE the stateless 2026-07-28 spec; that stays supported through a twelve-month deprecation window, so there is nothing to change. If you write your OWN MCP server, read the protocol section in docs/LAYERS.md first: an unbounded mcp>=x pin now pulls SDK 2.x, which removed mcp.server.fastmcp and breaks fresh installs.
3. In-graph Claude nodes: Claude as a step INSIDE a workflow (prompt enrichment, vision QA on the output).
4. Node-building skills: comfyui-node-* (V3 API) for when we write or modify a custom node.

docs/LAYERS.md explains each; install.ps1 / install.sh wires them up.

The client (no extra deps, stdlib only)

comfy_client.py lives next to this SKILL.md. Import and use:

import sys; sys.path.insert(0, r"<this skill dir>")
import comfy_client as c
c.alive()                                  # True if the API answers (override host with COMFY_HOST env)
c.run("path/to/workflow_api.json",
      overrides={"6.text": "a cinematic dragon, dark studio light", "3.seed": 12345},
      outdir=r"...\assets")                # queues, waits, downloads -> returns saved file paths

API surface: alive(), run(workflow_path, overrides, outdir, timeout), and the pieces queue(workflow), wait(prompt_id), download_outputs(rec, outdir), apply_overrides(wf, overrides). Override keys are "<nodeId>.<inputName>" (node ids and input names come straight from the workflow JSON). The MCP driver (Layer 2) does the same and more; use it when available, fall back to this client otherwise.

Workflow JSON (API format)

ComfyUI runs the "API format" graph: a dict { "<nodeId>": { "class_type": "...", "inputs": {...} }, ... }. To get one: in ComfyUI enable Settings -> Enable Dev mode Options, build the graph, then Save (API Format). Official starting graphs: Workflow -> Templates browser (per model). Save those as API Format, then parameterize.

To parameterize a graph, read it and find:

• the positive prompt: a CLIPTextEncode node, override .text.
• the seed: a KSampler / sampler node, override .seed (use a varied seed per call; do not hardcode).
• dimensions: an EmptyLatentImage / EmptySD3LatentImage node, override .width / .height.
• steps/cfg/sampler/scheduler: on the sampler node. Keep the template's values unless asked, they are model-tuned.

Template library (the SOURCE OF TRUTH, mix and match)

The official Comfy-Org workflow templates are the source of truth for how to do any task in ComfyUI. The kit clones them (sparse) to a local folder and builds a compact lookup index. Default location set by the installer; record it in the machine block above. Master index: templates/_quick_index.json (name -> title, category, models, tags, mediaType, vram, description), regenerate with shared/tools/gen_quick_index.py. Update: git pull in the clone, then rerun the generator.

Flow: read _quick_index.json, find the template whose name/models/tags match the request, read THAT one templates/<name>.json, parameterize it. New templates use SUBGRAPHS: the real pipeline is inside definitions.subgraphs[0], exposed params (text, width, height, seed, steps, model names) are in subgraphs[0].inputs, traced to inner nodes via the outer node's properties.proxyWidgets. Mix and match the blueprints/ (reusable subgraph bricks: text_to_image_z_image_turbo, image_to_video_ltx_2_3, image_upscale_z_image_turbo, remove_background_birefnet, ...).

widgets_values are ORDER-based, no field names: KSampler = [seed, control_after_generate, steps, cfg, sampler, scheduler, denoise]; EmptySD3LatentImage = [width, height, batch]. Model filenames must match installed files exactly. Validate node types/inputs against /object_info/<NodeType> before writing a graph.

Compose a NEW workflow from pieces (assemble + wire it correctly)

When no single template fits, BUILD one by chaining pieces. The skill is for assembling, not only running.

1. Decompose the task into stages, one brick per stage, e.g. text-to-image -> upscale -> image-to-video -> add audio. Pick a template or a blueprints/ subgraph for each stage (match via _quick_index.json / blueprint names), and read each one to see its real input and output nodes.

2. Know how nodes connect (the key mechanic).

• API format: every input is EITHER a literal value OR a reference to another node's output, written as a 2-item list ["<sourceNodeId>", <outputSlotIndex>]. To run stage B after stage A, set B's input to ["<A_id>", <slot>], where <slot> is the index of A's matching output. Example: feed a decode's IMAGE into an upscaler -> "image": ["8", 0] (node 8, output 0).
• GUI format: connections live in the top-level links array; each link is [link_id, src_node, src_slot, dst_node, dst_slot, type], and each node's inputs[].link / outputs[].links carry those link ids. Write THIS to show the graph in the canvas (the bridge); write the API form to run.

3. Match types, or convert. Every output and input has a TYPE: IMAGE, LATENT, MODEL, CLIP, VAE, CONDITIONING, AUDIO, MASK, CONTROL_NET, ... You may ONLY connect matching types. Read each node's input + output types from /object_info/<NodeType> (input.required / output / output_name). If a seam's types differ, insert a converter: VAEEncode (IMAGE -> LATENT), VAEDecode (LATENT -> IMAGE), CLIPTextEncode (text -> CONDITIONING), ImageScale / an upscaler for size. Never wire an IMAGE into a LATENT input.

Common node I/O (memorize these; for anything else read /object_info/<NodeType>). A node's WIDGETS are values you set; its INPUT SLOTS must receive the matching TYPE from another node's OUTPUT. You cannot feed text into a LoRA input, or a MODEL into a text box.

• CheckpointLoaderSimple -> out: MODEL, CLIP, VAE. (Flux/newer split loaders: UNETLoader -> MODEL ; DualCLIPLoader/CLIPLoader -> CLIP ; VAELoader -> VAE.)
• LoraLoader: in MODEL + CLIP (+ name/strength widgets) -> out MODEL, CLIP. A LoRA is applied ONTO the MODEL+CLIP stream, never wired as text.
• CLIPTextEncode: in CLIP + text widget -> out CONDITIONING. Your prompt becomes CONDITIONING here; downstream nodes want CONDITIONING, not raw text.
• EmptyLatentImage / EmptySD3LatentImage: widgets only -> out LATENT.
• KSampler / KSamplerAdvanced: in MODEL + positive CONDITIONING + negative CONDITIONING + LATENT (+ seed/steps/cfg/sampler/scheduler/denoise widgets) -> out LATENT.
• VAEDecode: in LATENT + VAE -> out IMAGE. VAEEncode: in IMAGE + VAE -> out LATENT.
• ControlNetLoader -> CONTROL_NET ; ControlNetApplyAdvanced: in CONDITIONING + CONTROL_NET + IMAGE -> out CONDITIONING.
• LoadImage -> IMAGE, MASK ; SaveImage / PreviewImage: in IMAGE. Basic txt2img stream: loader -> (LoraLoader) -> CLIPTextEncode x2 (pos/neg) -> KSampler (+ EmptyLatentImage) -> VAEDecode -> SaveImage. Before building, also check KNOWN_ISSUES.md (next to this file or docs/KNOWN_ISSUES.md) and ADVANCED.md for current bugs and workarounds, so you do not wire around a known-broken path.

4. Merge graphs cleanly. Splicing two templates: renumber one graph's node ids so they do not collide; SHARE the loaders (one CheckpointLoader / UNETLoader / VAELoader / CLIPLoader feeding both stages, do not duplicate the same model); then wire the seam (stage A's final output -> stage B's first input). Keep each model's own VAE / encoder with it (a Wan VAE is not an SDXL VAE; LTX bundles its VAE in the checkpoint).

5. Validate before running. Check: every class_type exists in /object_info; every input is a literal or a [node, slot] ref to an existing node; every seam's types match; model filenames exist locally; and the graph has at least one INPUT node carrying the user's intent AND at least one OUTPUT/save node wired to the final tensor (SaveImage / SaveAudio / SaveVideo / VHS_VideoCombine, or a PreviewImage). API / partner nodes (Kling, Nano Banana, Veo, Gemini, ...) often emit a tensor but include NO save node by default - add and wire one, or the job runs "successfully" and produces nothing retrievable, wasting the compute. Then run SMALL / low-res FIRST to confirm the wiring, before the full render. Emit both formats: GUI to show in the canvas, API to run. When unsure of a node's exact inputs/outputs, query /object_info/<NodeType> rather than guessing.

Shared workflows + model shootout (pick the best model for a look)

Beyond the named template library, ComfyHub hosts thousands of community-shared workflows at comfy.org/workflows/<hash>. Any ComfyHub share downloads as plain JSON from a predictable URL: https://comfy.org/workflows/download/<hash>.json. So you can grab any shared workflow on demand, then read or run it. Helper: python shared/tools/fetch_workflow.py <hash> <outdir> (stdlib). The <hash> is the id in the share URL. Note: cloud.comfy.org/?share=<hash> links are Comfy Cloud only and are NOT downloadable this way (open in Comfy Cloud and export from the canvas).

Model shootout (which model is best for THIS prompt): the template library already ships a comparison grid, templates-all_in_one-image_edit_models ("1 input and multiple editing model comparison"): it fans one input image through 7 image-edit models at once (Flux.2 Dev/Klein, GPT-Image-1.5, Grok, Nano Banana Pro, Qwen-Image-Edit, Seedream) and saves each output side by side, so you pick the best look before committing. For video, the community "Adjustment Frame" share (hash 7dca0438edf4) compares video backends (Grok/Kling/Veo/Seedance/Wan2.2/LTX-2). Run small / low-res first, compare, then scale up the winner. This pairs with the per-model recipes below and the hardware-aware fit check.

Real production graphs to study: Comfy-Org/creative-campus (github.com/Comfy-Org/creative-campus) collects the actual workflows from Comfy Education Initiative case studies, real graphs from award-winning artists (e.g. Xindi Zhang's Song of Drifters, a Student Academy Award film: SD1.5 style transfer with IP-Adapter + ControlNet, plus a 3D + AI morphing graph). Open and study them for production technique. Link-and-study only (no license file; shared with the artists' permission), so reference it, do not bundle the JSONs.

Staying current (new models and workflows)

ComfyUI ships new models constantly, and they land in the template library first. To see what is new: git pull the templates clone and regenerate the quick index (gen_quick_index.py), then DIFF the model list (names not seen before = new models / new templates). Also read the announcements RSS at https://blog.comfy.org/feed. The kit ships shared/tools/check_updates.py, which does all of this in one command (pull + diff + RSS). When a genuinely new generative model appears without a recipe, research its OFFICIAL prompting (maker docs / model card / docs.comfy.org) and add it to MODELS.md in the same format; a new utility/upscaler goes to the Enhancement section. Do NOT scrape LinkedIn (auth-gated, anti-scraping, ToS); the blog RSS and the templates repo carry the same news, machine-readable. Full loop: the kit's docs/UPDATING.md.

Per-model prompting (the mega-brain): READ before prompting a named model

Every generative model has its own dialect. SDXL wants comma tags, FLUX wants natural-language sentences, video models want camera + motion direction, audio models want genre/tempo/instruments, and negative-prompt support varies (FLUX and many turbo models ignore or break on negatives). The kit ships a per-model prompting reference, MODELS.md (next to this file), distilled from OFFICIAL sources: each maker's docs / model cards, docs.comfy.org, and the anthropic-claude node's per-model templates.

Auto-pull rule, and it is TWO reads: when a specific model is named in the request, the workflow, or the chosen template, open MODELS.md, find the model in its "every model with a recipe" table, then READ THAT FAMILY FILE under MODELS/ BEFORE writing the prompt. Follow its prompt structure, its negative-prompt rule, and its settings. Never carry one model's style to another.

MODELS.md is an index, not the reference. It was one file until the 2026-08-06 audit measured it at 174 KB with 57% of entries past the point where a read stops returning content, which made this rule silently no-op for most models while looking like it had worked. Each family file now reads whole in one call.

MODELS.md covers (image) FLUX.1/.2 + Kontext, Z-Image-Turbo, Qwen-Image/Edit, SDXL, SD1.5, SD3.5, HiDream, Ideogram, Nano Banana Pro/2, Seedream 4.x/5 Lite/5 Pro (incl. 5.0 Pro Layer Separation), Qwen Image 3.0 Pro (partner nodes shipped in core v0.32.0), Recraft, GPT-Image, Grok (incl. Grok Imagine Image 2.0 and the V2 edit node), Reve (deprecated in core v0.31.0), Kandinsky, BRIA, OmniGen, Chroma, Krea (incl. the Turbo image-style-reference LoRA on core nodes), ERNIE-Image, Mage-Flow (Microsoft 4B, native-resolution, MIT); (image edit) FLUX Kontext, Qwen-Image-Edit, FireRed, LongCat, ChronoEdit, JoyAI Image Edit, Mage-Flow-Edit; (video) Wan 2.1-2.7 (incl. Uni3C camera-trajectory ControlNet), Wan Animate 2 (local character animation, no pose extraction), LTX-2.5 (open weights AND API partner nodes, day-0 in core v0.32.0), LTX-2.3 / 2 Pro (its API nodes now deprecated), Hunyuan Video, SVD, Kling, Veo, Sora, FLUX 3 Video (BFL, with synchronized audio), Seedance (1.0 / 1.5 Pro / 2.0 / 2.5), Luma, Runway, MiniMax (incl. H3, API + local open weights), PixVerse, Vidu, Pika, Sync 3 (lip sync), HeyGen (avatar video, talking photo, video translate, TTS), HappyHorse, HuMo, SCAIL-2; (audio) Stable Audio, ACE-Step, MiniMax Music 3 (open-weight full songs with vocals, core v0.33.1), ElevenLabs, ChatterBox, Seed Audio, Sonilo; (3D) Hunyuan3D, Tripo, Rodin, Meshy; (newer/niche) Capybara, Bernini-R, Anima (+ ControlNet-LLLite control and inpainting patches), NewBie, PixelDiT, Ovis-Image, Lens, Quiver.

Talking-head routing: for a still portrait plus an audio track, Sync 3 and HeyGen Talking Photo both apply; prefer Sync 3 when the job is purely lip-sync fidelity on footage you already have, and HeyGen when you need the model to also SPEAK a script (its text-to-speech and voice library are built in) or to present as a reusable avatar. Neither takes a scene prompt.

It also has an Enhancement and utility section (not prompt-driven, use as pipeline steps with settings not prompts): upscale/restore/interpolation (Real-ESRGAN, SUPIR, SeedVR2, FlashVSR, Topaz, Magnific, FILM, RIFE) and segmentation/depth/pose/conditioning (SAM3, BiRefNet, Depth Anything, DWPose, MoGe, IP-Adapter, LivePortrait, Mediapipe) and video object-removal (VOID). For any model not detailed there, the template library + /object_info is the fallback, and the matching official doc link is the source.

In-graph Claude nodes (Layer 3): pick the right one

Three Claude nodes can exist after install; they differ by billing and purpose (see docs/NODES.md):

• AnthropicClaudeNode (category LLM/Anthropic, community, your own key), 40+ templates that rewrite a prompt for a specific model (Ideogram 3, LTX 2.5, LTX 2.3 / LTX 2 Pro, Wan 2.1 & 2.2, FLUX, Nano Banana, Veo 3, Sora 2, ...). Vision + extended thinking. Needs CLAUDE_API_KEY env. The workhorse for autonomous in-graph prompt enrichment.
• ClaudeNode (category partner/text/Anthropic, official Comfy-Org), billed via Comfy.org credits, no own key. Models up to the latest Opus. Fallback path.
• ClaudeCustomPrompt (Claude Prompt Generator), simple, api_key as a string input.

You only NEED a Claude node when a graph must enrich prompts WITHOUT Claude in the loop (e.g. an unattended auto-hero pipeline). When you are already driving, write the prompt yourself, it is better and free.

Build a workflow AND show it in the owner's GUI (bidirectional bridge)

The owner may want to SEE the graph Claude builds, in their own ComfyUI canvas, and tweak it. The bridge is the GUI workflows folder, which both sides read and write: <ComfyUI>/user/default/workflows/.

Two JSON formats, keep both in mind:

• GUI format (what the canvas loads and "Save" produces): top-level nodes (each with id, type, pos, size, widgets_values, inputs, outputs), links, groups. Write THIS to the workflows folder so the owner can OPEN and see the graph. Auto-layout nodes in left-to-right columns (loaders -> encode -> sampler -> decode -> save) with a Group box per stage and a per-column y-cursor so nodes never overlap. See "Lay the graph out cleanly" below for the exact discipline.
• API format (what /prompt runs): { "<id>": {class_type, inputs} }. Send THIS to run headlessly.

Flow:

• Claude builds -> write the GUI-format .json to user/default/workflows/<name>.json -> tell the owner to refresh the built-in Workflows sidebar (folder icon) and open it -> he sees exactly what Claude built.
• Owner builds/edits -> Save (API Format) into a shared workflows/ folder -> Claude reads and runs it.

Lay the graph out cleanly (structured blocks, no overlap)

A graph that piles nodes at 0,0 or lets them overlap is unusable in the canvas. Lay it out like a real pipeline. Every node carries pos:[x,y] + size:[w,h]; each block is a groups entry {title, bounding:[x,y,w,h], color}. COMPUTE positions, never eyeball them.

• Columns = stages, strictly left to right. One pipeline stage per column (loaders -> conditioning -> sample -> decode -> save -> post). Column x = x0 + col * COL_W, COL_W = widest node width + 80 (~360 typical). Data never flows backward (no right-to-left wire).
• Per-column y-cursor = zero overlap. Stack a column top-down: y = y0; place a node; then y += node_h + 60. The next slot always clears the previous node's full height, so nodes in a column cannot overlap, and COL_W >= widest + 80 clears them horizontally. Read real size from the template (assume ~[320,200], taller for KSampler / CLIPTextEncode). Never give two nodes the same pos.
• One Group box per stage (this is what makes it read as blocks). After placing a stage's nodes, add a group whose bounding wraps them with padding: [minX-30, minY-50, (maxX+w)-minX+60, (maxY+h)-minY+80] (extra top room for the title bar). Title by stage ("Load models", "Conditioning", "Sample", "Decode + Save", "Upscale", "Image to Video"); color-code (loaders grey, conditioning blue, sampler green, decode/save purple, post orange). Shared loaders sit in one group top-left, feeding every stage.
• Reroute long or crossing wires. When a shared output must reach a far column, insert Reroute nodes and run the wire along a horizontal gutter between groups instead of a diagonal across the graph. Kills the spaghetti look.
• Tidy pass before saving. Same node width per column, left edges aligned, seeds + savers last, no node outside its group box, no two group boxes overlapping. With the MCP, visualize_workflow_hierarchical renders the layout so you SEE overlaps before handing it to the owner.

Collapse a stage into one reusable node (Subgraphs)

ComfyUI Subgraphs (official since 2025-08; they supersede the old Group Nodes, kept only for back-compat) let you select a pile of nodes and fold them into a single super-node that exposes ONLY the few params you care about. This is the cleanest way to build and reuse pipeline bricks: a tested 20-node upscale or video stage becomes one node with 3 knobs, nestable into a bigger pipeline.

In the GUI (tell the owner, or do it yourself when driving):

• Collapse: select the nodes (plus groups and reroutes), click the subgraph icon in the toolbar. ComfyUI auto-wires the boundary from the selection's external inputs/outputs.
• Edit inside: double-click the subgraph's empty area to enter; Esc or the top nav bar to exit (the nav bar shows the nesting level).
• Expose only what matters: the Edit Subgraph Widgets button (parameters panel) reorders and shows/hides widgets without entering; in edit mode, right-click a boundary slot to rename/delete/disconnect, and the labeled default slot adds a new input/output. Surface seed/steps/cfg/prompt, hide the rest.
• Make it a reusable brick: Add Subgraph to Library (the publish/book icon, ComfyUI v1.27.7+) turns it into a Subgraph Blueprint, searchable and draggable like any node. This is exactly what this kit's blueprints/ bricks are.
• Nest subgraphs inside subgraphs for hierarchical pipelines; Unpack subgraph (right-click or the selection toolbox) reverts it to raw nodes.

When BUILDING the JSON yourself (not clicking): the inner graph lives in definitions.subgraphs[]; the outer SubgraphNode exposes params through properties.proxyWidgets and boundary I/O through the subgraph's input/output nodes (see the template-reading note above). Ship one clean brick per stage instead of 20 loose nodes. Sources: docs.comfy.org/interface/features/subgraph ; blog.comfy.org/p/subgraph-official-release.

Creator-level depth: strengths, real limits, and advanced sequence work

The full creator-level reference (strengths, the real gotchas with workarounds, advanced sequence techniques, and a verified tool table with licenses) is ADVANCED.md (next to this file in the installed skill, or docs/ADVANCED.md in the repo). Read it for hard tasks. The load-bearing gotchas to remember even without opening it:

• Black/NaN images or a color/contrast shift after decode = the VAE. Use --fp32-vae (or --bf16-vae), decode once at the end, and a histogram/LAB match to restore the source plate. Never fp16 VAE for VFX.
• A custom node that never re-runs is the IS_CHANGED footgun: force a rerun with return float("NaN"). A seed change that does nothing = stale cache; bust an input.
• Per-generation model reload thrash or a 4090/5090 slowdown (early 2026+) is Dynamic VRAM: --disable-dynamic-vram if it hurts.
• Single-digit canvas fps on a huge graph is litegraph, not the backend: collapse into subgraphs, mute groups.
• Custom nodes carry real malware risk and break on core/numpy bumps: install only from verified authors, pin versions.

Advanced tasks the skill can now reason about (verified tools and recipes are in ADVANCED.md):

• Temporal stability / anti-flicker for sequences: native video model (Wan 2.2 + VACE / HunyuanVideo 1.5 / LTX-2) > context windows + FreeNoise > per-frame depth/pose ControlNet to lock structure > light RIFE + light deflicker. SD-era: unsampling (Flip Sigma, Euler, add-noise OFF) + flow attention. Fine texture/identity and window seams still flicker.
• PBR / material passes from footage: be honest, native temporally-stable PBR from a 2D sequence is NOT solved in 2026. Per-frame decompose (Apache-2.0 Marigold-IID + StableNormal + StableDelight) + optical-flow temporal smoothing is the realistic path; rgb2x / CHORD are higher-fidelity but noncommercial; UniRelight is the only true temporal method (albedo-only, noncommercial, no node); TRELLIS.2 (MIT) gives real PBR but on a 3D mesh, not per-frame.
• Max detail + precision: tiled refine (Ultimate SD Upscale / Tiled Diffusion) + ControlNet Tile for seams, Detail Daemon / PAG / FreeU for micro-detail, dpmpp_2m + Karras ~20-35 steps, 32-bit EXR sequence I/O (HQ-Image-Save, CoCoTools) + sRGB/Linear conversion for VFX. Per-frame detail vs cross-frame stability is a real tradeoff (SeedVR2 batch >= 5, or lock structure + vary only fine detail).
• High-detail matting (hair / fur / semi-transparent / motion blur): multi-stage - coarse select (SAM3 / BiRefNet) -> trimap -> alpha matte (ViTMatte / SDMatte / Matte-Anything) -> edge refine (LayerStyle MaskEdgeUltraDetailV2). Video = temporal model MatAnyone2 (needs a SAM2/SAM3/SeC keyframe mask; NTU research license) or RVM for clean humans. The official library ships remove_background_birefnet (image) + SAM3 segmentation, but NO free local temporal video matte (the video-matte templates are paid Bria API). Full recipe in ADVANCED.md.
• Keep the two formats in sync: build once, emit both. Validate node names and inputs against /object_info/<NodeType> before writing, so the graph is not red/broken when he opens it.

No extra "agent panel" node is required for this; the built-in Workflows sidebar is the bridge. (The comfyui-mcp ecosystem has an optional live-streaming panel; it is polish, not a requirement.)

Where models live, and how to download one (DETECT, do not assume)

ComfyUI reads models from one or more model roots. On a source install it is <ComfyUI>/models/<type>/. On Comfy Desktop the active root is usually a SHARED folder set via extra_model_paths.yaml, NOT <ComfyUI>/models. Always detect the real root before downloading; a file in the wrong folder is invisible to ComfyUI.

Detect the real model root first:

• Read the ComfyUI startup log: it prints Adding extra search path <type> <PATH> for every model type, plus Setting output/input directory to: .... Those PATHs are the truth (MCP get_logs, or the Desktop log file).
• Or read <ComfyUI>/extra_model_paths.yaml (and the .example).
• Or ask the running server: /object_info/CheckpointLoaderSimple, /UNETLoader, /VAELoader, /CLIPLoader list the files currently visible, which confirms the folder is wired after you drop a file in.

Model type -> subfolder (under the detected root):

• diffusion / UNET single-file model -> diffusion_models (sometimes unet)
• full checkpoint (model+clip+vae bundled; some video like LTX ship this way) -> checkpoints
• text encoder / CLIP (T5, umt5, gemma, clip_l, llava) -> text_encoders (older installs: clip)
• VAE -> vae · LoRA -> loras · upscaler (ESRGAN, etc.) -> upscale_models
• ControlNet -> controlnet · IP-Adapter -> ipadapter · CLIP vision -> clip_vision

How to download (Desktop-safe):

• Direct download is most reliable: curl -fL -C - -o "<root>/<type>/<filename>" "<url>". Use the official Comfy-Org repackaged Hugging Face repos (.../resolve/main/... direct links). -C - resumes a partial file. Big models (tens of GB) are fine to run in the background; verify final size after.
• COMFYUI_PATH gates a whole family of MCP tools, not just downloads. Anything that reads the install's filesystem rather than its HTTP API fails with COMFYUI_PATH is not configured when it is unset: list_output_images is the one you hit first, since it is the natural way to find what you just rendered. Confirmed on a live run 2026-08-06. The workaround needs no configuration: pull the filenames from GET /history/<prompt_id> and fetch the bytes from GET /view?filename=...&type=output&subfolder=..., which is pure HTTP and always works. Set COMFYUI_PATH to the real ComfyUI root if you want the filesystem tools as well.
• The MCP download_model works ONLY if the MCP server has COMFYUI_PATH set, and it writes to COMFYUI_PATH/models/<type>, which on Desktop is usually NOT the shared root, so files can land where ComfyUI cannot see them. Prefer direct download to the detected root (or set COMFYUI_PATH to the real root first).
• Gated models (e.g. Stability Stable Audio) need a Hugging Face login + license acceptance: ask the owner to accept the license and place the file, or provide an HF token for an authenticated download.
• The exact file set per model (diffusion model + text encoder(s) + VAE, and which folder each goes in) is on the model's docs.comfy.org/tutorials/... page; follow it rather than guessing quant levels or filenames.
• After download, confirm ComfyUI sees it: re-query /object_info/<LoaderNode>. Most model folders refresh live; a brand-new subfolder may need a Workflows-sidebar refresh.

Pick a model variant that fits THIS machine (hardware-aware, recommend before downloading)

Before installing or downloading a model, size it against the real hardware, then RECOMMEND, do not download blindly. Detect three numbers and compare them to the model's footprint.

Detect (reuse the bootstrap machine block, or refresh):

• VRAM per GPU (free + total): MCP get_system_stats / health_check, or GET /system_stats (devices[].vram_free / vram_total). With two cards, note each separately.
• System RAM (free + total): same /system_stats. RAM matters for weight offloading and spill.
• Free disk on the MODEL drive: check the drive that holds the detected model root (not the system drive). df -h "<model root>" in Git Bash, or the platform equivalent. Downloads run to tens of GB; never start one that will not fit.

Estimate a model's footprint:

• VRAM needed roughly equals the diffusion model's on-disk weight size, plus VAE + text encoder + activations (rule of thumb: weights size + ~2-6 GB headroom; video models need much more for the latent frames).
• Precision ladder, smaller fits more: bf16/fp16 (full) > int8-convrot and fp8 (~half) > GGUF Q8 > Q6 > int4-convrot and Q4 (smallest). MODELS.md lists the recommended variant and any VRAM note per model.
• At the half-size tier, prefer int8-convrot over fp8 unless a model ships only fp8. It is native since core v0.27.0, runs faster than fp16, and matches or beats fp8 on quality, with Turing and Ampere explicitly supported (RTX 20xx / 30xx). Full method, the ConvRot paper, the conversion tool and the known int8 bugs are in docs/ADVANCED.md; read it before choosing a variant, because this one line is the summary, not the rule.
• "Pruned" is a fourth axis, not a rung on this ladder. A pruned checkpoint drops weights rather than narrowing their precision, so it stacks with any of the above and its quality cost is model-specific. nvfp4 is a Blackwell-generation format (RTX 50xx); on Ampere it is not an option.
• Download size on disk roughly equals the sum of every file (model + encoder(s) + VAE). Sum them first.

Decide and recommend:

• Fits one card with headroom -> use it, full precision.
• Slightly over one card -> ComfyUI weight offloading (weights in RAM, streamed to VRAM) or the fp8 variant; recommend that, do not force bf16.
• Far over one card but fits across both -> MultiGPU DisTorch layer-split (only then; it is slower).
• Over total VRAM even split, but RAM is large -> CPU/RAM offload (slow) or a GGUF Q4/Q5; recommend the quant.
• Not enough VRAM at any precision, or not enough free disk -> DO NOT download. State the exact shortfall (e.g. "LTX-2.3 fp8 is ~28 GB on disk and wants ~24 GB VRAM, but the model drive has only 12 GB free") and the cheapest fix (smaller variant, free disk, or skip).
• Coordinate with other GPU users (Ollama): free VRAM may be held, see the VRAM section.

Always, before a download: compare the summed download size to the model drive's free space, and the model's VRAM need to the card it will run on. State the verdict so the owner sees the reasoning, not just a result: "fits, downloading" / "too big for 24 GB, using fp8" / "only 10 GB free on E:, cannot fit ~28 GB, stopping".

Using multiple GPUs (it is NOT like a layer-split LLM server)

One generation runs on ONE card; ComfyUI does not auto-spread a single small job across cards. Wins from the MultiGPU nodes (SelectModelDevice, SelectCLIPDevice, `SelectVAEDevi

…(truncated)