Comfyui Operator

comfyui-operator

This skill provides native integration to operate a local instance of ComfyUI. It allows you to:

1. Guarantee ComfyUI is running.
2. Interrogate the installation for available Workflows, LoRAs, and Checkpoints.
3. Queue prompts (API workflows) with dynamic JSON overrides and download the resulting images.

Configuration & Setup

All scripts rely on the requests and websocket-client Python libraries. Depending on your Python environment, you may need to install them:

pip install -r requirements.txt

By default, all scripts assume ComfyUI is installed at C:\ComfyUI_windows_portable. IMPORTANT: If the default directory does not exist, ask the user for the correct ComfyUI installation path before proceeding.

Interaction Guidelines

When a user asks you to generate an image using this skill, you must follow these conversational rules:

1. Vague/General Requests: If the user simply says "Generate an image" without specifying a character, concept, or workflow:

   • DO NOT guess a workflow.
   • Run list_resources.py immediately to discover available API-format workflows.
   • Present the list of found workflows to the user and ask them which one they would like to use.

2. Output Locations (CRITICAL): When invoking scripts that generate files (list_resources.py or generate_image.py), you MUST explicitly specify the user's active workspace directory using the --output and --output_dir arguments. Do not rely on script default paths.

   • For list_resources.py, always save the output to <active_workspace>/.comfyui/resources.json unless the user explicitly specifies a different path.
   • For generate_image.py, always save the output images to <active_workspace>/.comfyui/outputs/ unless the user explicitly specifies a different saving location.

3. Specific Requests & Fuzzy Matching: If the user asks for a specific subject (e.g., "Generate an image of Nemu" or "ねむの画像を生成して"):

   • Run list_resources.py to get the list of workflows.
   • Perform fuzzy matching to find a workflow that matches the requested subject, accounting for orthographical variants (e.g., "Nemu", "nemu", "ネム", "ねむ" should all match a file like os-nemu-api.json).
   • If a plausible match is found, proceed with generating the image using that workflow directly (unless disambiguation is needed between multiple close matches).

Scripts

1. start_comfyui.py

Ensures that the ComfyUI server is running and the API is reachable.

Usage:

python scripts/start_comfyui.py [--comfyui_path "path/to/comfyui"]

It returns silently if ComfyUI is already running. Otherwise, it will launch the startup batch file or Python script in the background and poll until http://127.0.0.1:8188/system_stats responds.

2. list_resources.py

Lists available models (Checkpoints, LoRAs) and Workflows saved in the API JSON format to help formulate generation parameters.

Usage:

python scripts/list_resources.py [--comfyui_path "path/to/comfyui"] [--output "resources.json"]

Outputs a JSON block representing the contents of the models directories and user/default/workflows. Use the --output argument to save directly to a file and avoid console encoding issues. By default, it will save to resources.json in the current working directory.

3. generate_image.py

Generates an image by loading a base JSON workflow, applying targeted node overrides over the WebSocket API, and saving the final image.

Usage:

python scripts/generate_image.py \
    --workflow_path "path/to/base_api_workflow.json" \
    [--output_dir "path/to/save/images"] \
    [--overrides_file "path/to/overrides.json"] \
    --overrides '{"3": {"inputs": {"seed": 12345}}, "6": {"inputs": {"text": "A beautiful sunset..."}}}' \
    [--comfyui_path "path/to/comfyui"] \
    [--server "127.0.0.1:8188"]

Tip: Due to Windows terminal quoting and escaping issues when passing JSON strings via --overrides, it is highly recommended to write complex overrides to a temporary JSON file and pass it using --overrides_file temp.json.

Note: The script will auto-detect whether the server is running on port 8188 or 8000. Note: The default output directory is the current working directory.

Understanding Overrides

The --overrides argument is a JSON string. The keys are the Node IDs from the workflow file (often represented as strings of numbers, e.g., "3", "6").

• KSampler nodes typically have ID "3". To change the seed: {"3": {"inputs": {"seed": <random_number>}}}
• CLIP Text Encode (Positive Prompt) typically has ID "6". To change the prompt: {"6": {"inputs": {"text": "<your_prompt>"}}}

⚠️ CRITICAL SEED SAFETY RULE: By default, DO NOT override or change the seed value in the base workflow. You should only inject a random seed if:

1. The user explicitly requests a new/different/random seed.
2. The user requests multiple images (batch generation) for the exact same prompt, in which case you must provide a different seed for each generation request to prevent identical outputs.

⚠️ CRITICAL PROMPT SAFETY RULE: When overriding a text prompt (e.g., node "6"), DO NOT overwrite the entire existing prompt blindly. Workflows often contain critical baseline tags (e.g., character names, style tags, quality tags like masterpiece, best quality).

• If you need to make changes, read the original workflow JSON first and append your new keywords to the existing text string.
• If the user asks for a significant departure from the base prompt, ask for their confirmation before performing a destructive overwrite.

Process:

1. Wait for ComfyUI to be running using start_comfyui.py.
2. Ensure you have a valid workflow_path pointing to a ComfyUI JSON file exported in API format.
3. Formulate the --overrides string to inject dynamic parameters (remembering to append, not replace, base text prompts unless specified).
4. Run generate_image.py and capture the saved image file paths.

Resources

This skill includes example resource directories that demonstrate how to organize different types of bundled resources:

scripts/

Executable code (Python/Bash/etc.) that can be run directly to perform specific operations.

Examples from other skills:

• PDF skill: fill_fillable_fields.py, extract_form_field_info.py - utilities for PDF manipulation
• DOCX skill: document.py, utilities.py - Python modules for document processing

Appropriate for: Python scripts, shell scripts, or any executable code that performs automation, data processing, or specific operations.

Note: Scripts may be executed without loading into context, but can still be read by Claude for patching or environment adjustments.

references/

Documentation and reference material intended to be loaded into context to inform Claude's process and thinking.

Examples from other skills:

• Product management: communication.md, context_building.md - detailed workflow guides
• BigQuery: API reference documentation and query examples
• Finance: Schema documentation, company policies

Appropriate for: In-depth documentation, API references, database schemas, comprehensive guides, or any detailed information that Claude should reference while working.

assets/

Files not intended to be loaded into context, but rather used within the output Claude produces.

Examples from other skills:

• Brand styling: PowerPoint template files (.pptx), logo files
• Frontend builder: HTML/React boilerplate project directories
• Typography: Font files (.ttf, .woff2)

Appropriate for: Templates, boilerplate code, document templates, images, icons, fonts, or any files meant to be copied or used in the final output.

Any unneeded directories can be deleted. Not every skill requires all three types of resources.