Autofigure

autofigure — AutoFigure / AutoFigure-Edit figure generation

Turns a figure idea (or a rough draft image) into a clean, editable vector figure. Two-stage pipeline, all local + subscription CLIs (no raw API keys required):

1. Step-1 raster — a journal-style draft PNG, by default from gpt-image-2 (OpenAI/Codex, via the hermes-gpt-image skill). nano-banana (Gemini) is a manual alternative. Skippable if you already have an image (a draw.io export, a screenshot, a hand sketch).
2. Vectorize (AutoFigure-Edit) — SAM3 segments the draft into regions, RMBG-2.0 cuts out transparent icons, and an LLM (gpt-5.5 via a local Codex shim) re-draws the whole thing as a placeholder-aligned editable SVG (final.svg). Then export an exact-size PDF.

Defaults — OpenAI via the Codex subscription (no raw API key): gpt-image-2 for the step-1 raster (gen.sh) and gpt-5.5 for the SVG re-draw (afe.sh, through the local shim → openclaw infer model run --model openai/gpt-5.5). nano-banana (Gemini) and SiliconFlow are optional, non-default alternatives — used only if you explicitly ask, or set SF_API_KEY.

Heavy assets (the afe_venv, sam3.pt, RMBG weights, the AutoFigure-Edit repo) live under AUTOFIG_HOME (default ~/apps/autofig_work). The scripts reference them; nothing is duplicated.

Workflow (follow every time)

1. Preflight once: bash ~/.claude/skills/autofigure/scripts/doctor.sh. If anything is ✗, see Setup below and stop until fixed.
2. Make / obtain the step-1 raster.
   • From text: bash scripts/gen.sh "<detailed figure prompt>" /tmp/fig_input.png Write a concrete, diagram-style prompt: state the layout ("left-to-right pipeline of N boxes with arrows"), each box's short label, palette, "flat vector, white background, crisp readable sans-serif labels, no clutter". gpt-image-2 spells short English labels well; keep dense diagrams to ≲15 labels or text may garble.
   • Or skip this and use an existing image as the input.
   • Read the raster to confirm it matches before vectorizing.
3. Vectorize: bash scripts/afe.sh <input.png> <out_dir> ["sam,prompts"] [svg_model]
   • Produces <out_dir>/final.svg (+ template.svg editable layout, icons/ assets, samed.png segmentation overlay, boxlib.json).
   • sam_prompt (optional) tunes what SAM3 looks for, e.g. "icon,box,arrow,text label,bottle".
   • Default SVG model is gpt-5.5 via the local shim (auto-started). To use SiliconFlow instead: SF_API_KEY=sk-... SVG_MODEL=Qwen/Qwen3-VL-32B-Instruct bash scripts/afe.sh ....
4. Preview: bash scripts/view.sh <out_dir>/final.svg /tmp/fig_preview.png then Read it. Check labels are correct/legible and nothing overlaps. If off, fix the prompt and redo step 2–3, or hand-edit final.svg/template.svg (it is plain SVG text).
5. Export PDF for the paper: bash scripts/svg2pdf.sh <out_dir>/final.svg figures/fig_x.pdf 10 (10 = width in inches; height auto from the SVG aspect; margin-free). Drop it into the paper with \includegraphics[width=\textwidth]{figures/fig_x.pdf}.

Commands (scripts/)

• install.sh — one-time backend install (venv + repos + SAM3 + RMBG + HF cache seed); see Setup.
• doctor.sh — verify the install is ready.
• gen.sh "<prompt>" <out.png> [landscape|square|portrait] — step-1 raster via gpt-image-2.
• afe.sh <input.png> <out_dir> [sam_prompt] [svg_model] — the AutoFigure-Edit vectorizer.
• svg2pdf.sh <in.svg> <out.pdf> [width_in] — exact-size, margin-free PDF.
• view.sh <in.svg|pdf> <out.png> [width] — PNG preview to Read.
• shim.py — local OpenAI-compatible server that bridges /v1/chat/completions (incl. images) to openclaw infer model run --model openai/gpt-5.5. afe.sh starts it on demand (port 8745) and leaves it running for reuse; stop it with pkill -f autofigure/scripts/shim.py.

Tips

• Editing the result: final.svg embeds the extracted icons; template.svg is the clean layout with labeled placeholders — easiest to tweak text/positions by hand, then re-export PDF.
• Dense, label-heavy method figures sometimes vectorize cleaner from a draw.io export fed straight into afe.sh (skip gen.sh) than from a text-generated raster.
• From-scratch SVG (no raster): AUTOFIG_HOME/run_af.py drives the original AutoFigure agent (text → SVG directly via an OpenAI-compatible LLM). Less reliable here than the Edit path; prefer gen.sh → afe.sh.
• Image generation draws on the ChatGPT/Codex (and Google, for nano-banana) subscription quota; the SVG re-draw uses Codex (gpt-5.5) or your SiliconFlow balance.

Setup (if doctor.sh reports ✗)

One-command install (builds the venv, clones the repos, installs SAM3, downloads the SAM3 + RMBG-2.0 weights, seeds the HF cache):

HF_TOKEN=hf_xxx bash ~/.claude/skills/autofigure/scripts/install.sh   # [AUTOFIG_HOME]

HF_TOKEN is required (briaai/RMBG-2.0 is gated — request access first, then make a read token). You still install the driver CLIs yourself: hermes (gpt-image-2), openclaw (gpt-5.5), and Google Chrome. install.sh is idempotent (skips anything already present).

What it lays down under AUTOFIG_HOME (~/apps/autofig_work):

• afe_venv — Python 3.11 venv with torch torchvision timm transformers kornia pillow cairosvg plus SAM3 installed editable (pip install -e from facebookresearch/sam3).
• sam3.pt — the SAM3 checkpoint, symlinked into the HF cache so offline mode finds it: ln -sf $AUTOFIG_HOME/sam3.pt ~/.cache/huggingface/hub/models--facebook--sam3/snapshots/<commit>/sam3.pt
• rmbg_local/ — a local copy of briaai/RMBG-2.0 (model.safetensors, config.json, birefnet.py).
• .hf_env — exports HF_TOKEN (+ optional FAL_KEY).
• AutoFigure-Edit/ and AutoFigure/ — the two repos.
• CLIs on PATH: hermes (gpt-image-2 + gpt-5.5 image desc), openclaw (gpt-5.5 SVG LLM), optional agy (nano-banana). Plus Google Chrome for SVG→PDF/PNG. Network note: keep HF_HUB_OFFLINE=1 (afe.sh sets it) — the HF Xet downloader hangs on some networks; seeding sam3.pt into the cache + local RMBG path avoids all HF downloads.