Comfyui

ComfyUI Skill

Drive ComfyUI for AI image/video generation, node-based workflow authoring, and distributed GPU compute via the Salad Cloud API. ComfyUI runs in an external Docker container reached over the Docker network — you talk to it through its HTTP API, never the local filesystem.

agentbox.toml's skills.media.comfyui_builtin = false and this skill's mcp.json entry "x-agentbox-gate": "env:ENABLE_COMFYUI_EXTERNAL" name the same fact: ComfyUI is not baked into the image, it runs externally.

Container architecture (read first)

ComfyUI runs in an EXTERNAL Docker container, not locally. When calling from Claude Code (its own container), pick the endpoint that matches where you are:

Networking rules that actually bite:

1. From inside the Claude Code container, localhost:8188 will not reach ComfyUI — use the comfyui Docker hostname.
2. You cannot read the container filesystem; retrieve outputs via /view?filename=...&type=output, not filesystem paths.
3. The comfyui container's volume mounts (output/input/models under /mnt/mldata/.../comfyui/...) are host-accessible only, not from here.

Check container status

From the Claude Code container, use the HTTP health check — this is the canonical liveness probe and needs no Docker access:

ping -c1 comfyui  # Should show IP like 172.18.0.X
curl -s http://comfyui:8188/system_stats | jq '.devices[0].name'

Docker-daemon diagnostics (no sudo). agentbox mounts the host Docker socket, so plain docker talks to the host daemon from inside the container. Do not prefix with sudo — it is blocked by no-new-privileges and will fail:

docker ps --filter "name=comfyui"
docker logs comfyui --tail 20

Prefer the comfyui:8188 HTTP check above; only fall back to these when the HTTP probe is inconclusive.

Quick path

To build or repair the local GPU service, follow references/sidecar-operations.md.

For resumable video jobs, use scripts/comfy_client.py; read references/agent-kit.md for graph validation and durable submission receipts. Local video model setup and workflows are documented in references/video-models.md. For a complete explainer of the current repository, start with codebase-video.

Generate a FLUX 2 image end-to-end with the bundled runnable script:

comfyui-generate \
  "A stunning landscape at golden hour, cinematic lighting" output.png

It submits the current FLUX 2 workflow to http://comfyui:8188, polls to completion, and downloads the PNG. For the manual step-by-step (health check → submit → poll → download), the one-liner variant, VRAM freeing, and the live model/node tables, see references/flux2-quickstart.md.

Capabilities

• Text-to-image (text2img) and image-to-image (img2img) generation
• Author and execute node-based workflows programmatically
• Video generation (AnimateDiff, CogVideoX, HunyuanVideo, LTX, Mochi, WAN, Cosmos)
• Manage models, LoRAs, checkpoints, VAEs
• Upscaling and post-processing
• Deploy to Salad Cloud for distributed GPU compute at scale

When to use this skill

• Generate AI images from text descriptions or transform image-to-image
• Design ComfyUI workflows programmatically
• Batch-process generation tasks
• Use or fine-tune LoRA models with FLUX/SD
• Generate AI video from text or images
• Deploy image generation at scale across distributed GPUs

When not to use

• 2D image manipulation (resize, crop, format convert) — use the imagemagick skill
• 3D modelling and scene creation — use the blender skill
• Video transcoding, editing, or audio extraction — use the ffmpeg-processing skill
• Diagrams, flowcharts, architecture visuals — use the mermaid-diagrams skill
• ML model training (classification, NLP, time series) — use pytorch-ml or flow-nexus-neural

References

Detail lives in references/ — pull the file that matches the task:

• references/flux2-quickstart.md — full FLUX 2 generate/monitor/download flow, one-liner, VRAM management, current model + node tables.
• references/api-and-parameters.md — local server operations, API endpoints, text2img parameter table, sampler/scheduler lists, output conversion, webhooks, error handling, directory layout.
• references/workflows.md — FLUX 2 and legacy FLUX 1 workflow JSON structures, supported image/video models, batch-generation Python.
• references/salad-cloud.md — Salad SDK deployment, the full recipe catalog (image/video/LLM), hardware recommendations, benchmarks, performance notes.

Bundled runnable assets in this directory:

• comfyui-generate — FLUX 2 one-shot generator (quick path above), on PATH from the agentbox-ops crate.
• anima_workflow.json / anima_import_workflow.json — working sample animation workflow graphs, kept as reference structures for generate.py.
• mcp-server/ — ComfyUI MCP server surface.

Integration with other skills

• imagemagick — post-process generated images
• ffmpeg-processing — assemble/transcode generated video
• blender — 3D-to-2D workflows
• pytorch-ml — custom model training

Related skills

• art for editorial illustration, infographics and comics via the Nano Banana image models; ComfyUI is the local-GPU FLUX/Stable Diffusion/video path.