Model-Scaffold Skill
Purpose
This skill stamps out a runnable PyTorch training repo for a medical-imaging task — --task
segmentation (U-Net), classification (CNN / timm backbone), detection (torchvision Faster
R-CNN / FPN), synthesis (Pix2Pix generator + PatchGAN), ssl (SimCLR encoder), or finetune
(transfer-learning a pretrained backbone with a frozen→unfrozen schedule + a provenance record) —
with the reproducibility guarantees baked in by construction — so the build is leakage-safe and
reproducible before a single epoch runs. It is the imaging analogue of how /analyze-stats generates
runnable statistical code: the generator produces the repo, you run the training on your GPU / Colab,
and the lane's deterministic gates verify the network-free parts.
It is the missing middle link in the lane: /architecture-zoo (choose) → model-scaffold (build)
→ /model-validation (validate the split / design) → /model-evaluation + /analyze-stats (metrics)
→ /write-paper + /check-reporting (publish). It integrates MONAI / nnU-Net / TorchIO (referenced
in the generated requirements.txt); it does not reimplement them.
When to use
- You have a data manifest (one row per image, with a patient/subject ID) and want a reproducible,
leakage-safe starting repo for a segmentation model.
- You want to fine-tune a pretrained backbone (transfer learning — the common clinician workflow:
a
timm / MONAI / MedSAM checkpoint adapted to your collected clinical data) with the freeze schedule,
discriminative learning rates, and pretrained-weight provenance recorded (--task finetune).
When NOT to use
- Auditing an already-trained model's validation design →
/model-validation.
- Held-out metrics / calibration / bootstrap CIs →
/model-evaluation then /analyze-stats.
- Choosing the architecture for the research question →
/architecture-zoo (when available).
- Reimplementing MONAI / nnU-Net → out of scope (the scaffold integrates them).
- LLM / MLLM evaluation →
/mllm-eval.
Workflow
Phase 1 — Prepare the manifest
A CSV with one row per image and a patient/subject ID column (patient_id / subject_id /
case_id), plus image and label path columns. The ID column is load-bearing: the split is done at the
patient level off this column.
Phase 2 — Generate the repo
python3 ${CLAUDE_SKILL_DIR}/scripts/scaffold.py \
--manifest <manifest.csv> --task segmentation --out model_repo --seed 42 \
--in-channels 1 --out-channels 1
# --task = segmentation | classification | detection | synthesis | ssl | finetune
# (out-channels = num classes for classification/finetune, target channels for synthesis)
# fine-tuning a pretrained backbone (transfer learning) on collected clinical data:
python3 ${CLAUDE_SKILL_DIR}/scripts/scaffold.py \
--manifest <manifest.csv> --task finetune --out model_repo --seed 42 \
--out-channels <num_classes> --from-pretrained timm:resnet50.a1_in1k
# emits PRETRAINED.md (provenance) + a frozen→unfrozen train.py with discriminative LRs;
# record the exact pretrained source so the fine-tune is reproducible.
This writes model_repo/ with config.yaml, model.py (the task's model — U-Net / CNN / Faster R-CNN
/ Pix2Pix / SimCLR encoder), dataset.py (reads the frozen split), losses.py (task-appropriate),
train.py, evaluate.py, requirements.txt,
REPRODUCIBILITY.md, methods_stub.md, and — the key artifact — splits/split_assignment.csv +
splits/split_seed.txt. The split is patient-disjoint by construction (a deterministic group split)
and the emitted code seeds every RNG, sets cuDNN deterministic, builds the training loader from the
train split only, and infers under model.eval() + torch.no_grad().
Phase 3 — Verify the build (network-free)
# this skill's own training-hygiene gate
python3 ${CLAUDE_SKILL_DIR}/scripts/check_training_hygiene.py --repo model_repo --strict
# the split-leakage gate (proves patient disjointness) — owned by /model-validation
Route the emitted splits/split_assignment.csv to /model-validation
(check_split_leakage.py --splits model_repo/splits/split_assignment.csv --strict) for the
patient-disjointness proof, and (optionally, locally with torch installed)
bash ${CLAUDE_SKILL_DIR}/scripts/scaffold_challenge/verify.sh to smoke the forward pass.
Phase 4 — Plug in your data and train
Implement dataset.py's _load_image / _load_label for your modality (DICOM / NIfTI / TIFF via
nibabel / pydicom / tifffile / TorchIO / MONAI transforms). For production, swap model.py for MONAI
UNet / SegResNet or an nnU-Net plan (see ${CLAUDE_SKILL_DIR}/references/training_guide.md). For a
fine-tuning repo (--task finetune), fill PRETRAINED.md and set the freeze schedule / discriminative
learning rates (see ${CLAUDE_SKILL_DIR}/references/finetuning_guide.md, which also covers MedSAM/SAM
adaptation and train-only diffusion augmentation). Run python train.py (best model selected on the
val split), then python evaluate.py (predictions on the test split, touched once).
Phase 5 — Validate, evaluate, publish
Hand off to /model-validation (validation-tier + comparator + metric-selection audit),
/model-evaluation + /analyze-stats (Dice + HD95/NSD with CIs), /make-figures, and /write-paper
(fill the methods_stub.md [VERIFY] placeholders) + /check-reporting (CLAIM 2024 / TRIPOD+AI). For
reproducibility-safe wiring of experiment tracking (W&B / MLflow), config / data / environment
versioning, and the MLOps reporting checklist, see ${CLAUDE_SKILL_DIR}/references/mlops_guide.md
(a wiring + reporting reference — it points to the frameworks, it does not replace them).
Runnability — honest contract
The generated repo is runnable, but runnability is not a CI guarantee. The default gates prove
the network-free properties (the emitted split is patient-disjoint + seeded; the emitted training code
is hygienic) by parsing the produced artifacts — no torch is executed. A torch forward-pass smoke
(build + forward shape + gradients flow + reproducible loss) is a self-skipping tier in the
challenge verify.sh and a documented local command; it is never counted as CI coverage of
runnability.
Anti-Hallucination
- Never fabricate training or evaluation metrics. The scaffold emits
[VERIFY] placeholders;
every number must come from the user's executed run and from /model-evaluation + /analyze-stats.
- Never emit a split that is not patient-disjoint or not seed-locked. The generator does this by
construction; do not hand-edit the split table to introduce overlap or remove the seed.
- Never claim the generated repo was trained or that it achieved a result — it is a starting point
the user runs.
- If a library API, default, or architecture detail is uncertain, flag
[VERIFY] and ask rather than
guessing.
Deterministic gates
scripts/scaffold.py — the generator (stdlib + numpy; deterministic given manifest + seed).
scripts/check_training_hygiene.py — AST linter: all RNGs seeded, cuDNN deterministic,
eval() + no_grad() inference, no training on a non-train split, and (fine-tuning) a
recorded pretrained-weight provenance when pretrained weights are loaded
(PRETRAINED_PROVENANCE_MISSING).
scripts/scaffold_challenge/verify.sh — the build → validate chain, network-free (torch tier
self-skips).
Boundaries
architecture-zoo (choose)
└─ model-scaffold (this skill: generate the reproducible repo)
├─ check_training_hygiene.py (training-code hygiene)
├─ model-validation (split-leakage proof + validation design)
├─ model-evaluation -> analyze-stats (metrics + CIs)
└─ write-paper + check-reporting (Methods stub -> compliant manuscript)
1---2name: model-scaffold3description: >4 Generate a reproducible, runnable PyTorch training repo for a medical-imaging task — segmentation,5 classification, detection, image-to-image synthesis, self-supervised pretraining, or fine-tuning a6 pretrained backbone (transfer learning) — the missing middle link between choosing an architecture and7 validating a trained model. Emits a patient-level seed-locked split as an auditable artifact, a8 task-appropriate model, train and evaluate scripts that seed every RNG and infer under eval mode, a9 config, requirements, a reproducibility record, and a Methods stub with VERIFY placeholders (no10 fabricated numbers). Fine-tuning mode adds a frozen-then-unfrozen schedule, discriminative learning11 rates, and a pretrained-weight provenance record. The reproducibility guarantees hold by construction,12 so the build is leakage-safe before any training runs. Integrates with MONAI, nnU-Net, TorchIO, timm,13 and torchvision — it does not reimplement them.14triggers: model scaffold, scaffold a model, training repo, PyTorch repo, build a model, train a model, fine-tune, finetune, transfer learning, pretrained backbone, MedSAM, SAM adaptation, segmentation, classification, detection, image synthesis, self-supervised, SimCLR, Pix2Pix, Faster R-CNN, U-Net, UNet, nnU-Net, MONAI, timm, torchvision, dataloader, train.py, patient-level split, reproducible training, seed everything, generate training code, medical imaging model15tools: Read, Write, Edit, Bash, Grep, Glob16model: inherit17metadata:18 source: https://skillmd.com/skills/aperivue/model-scaffold19---2021# Model-Scaffold Skill2223## Purpose2425This skill stamps out a **runnable PyTorch training repo** for a medical-imaging task — `--task`26**segmentation** (U-Net), **classification** (CNN / `timm` backbone), **detection** (torchvision Faster27R-CNN / FPN), **synthesis** (Pix2Pix generator + PatchGAN), **ssl** (SimCLR encoder), or **finetune**28(transfer-learning a pretrained backbone with a frozen→unfrozen schedule + a provenance record) —29with the reproducibility guarantees **baked in by construction** — so the build is leakage-safe and30reproducible before a single epoch runs. It is the imaging analogue of how `/analyze-stats` generates31runnable statistical code: the generator produces the repo, you run the training on your GPU / Colab,32and the lane's deterministic gates verify the network-free parts.3334It is the **missing middle link** in the lane: `/architecture-zoo` (choose) → **model-scaffold (build)**35→ `/model-validation` (validate the split / design) → `/model-evaluation` + `/analyze-stats` (metrics)36→ `/write-paper` + `/check-reporting` (publish). It **integrates** MONAI / nnU-Net / TorchIO (referenced37in the generated `requirements.txt`); it does not reimplement them.3839## When to use40- You have a data manifest (one row per image, with a patient/subject ID) and want a reproducible,41 leakage-safe starting repo for a segmentation model.42- You want to **fine-tune a pretrained backbone** (transfer learning — the common clinician workflow:43 a `timm` / MONAI / MedSAM checkpoint adapted to your collected clinical data) with the freeze schedule,44 discriminative learning rates, and pretrained-weight provenance recorded (`--task finetune`).4546## When NOT to use47- Auditing an already-trained model's validation design → `/model-validation`.48- Held-out metrics / calibration / bootstrap CIs → `/model-evaluation` then `/analyze-stats`.49- Choosing the architecture for the research question → `/architecture-zoo` (when available).50- Reimplementing MONAI / nnU-Net → out of scope (the scaffold integrates them).51- LLM / MLLM evaluation → `/mllm-eval`.5253## Workflow5455### Phase 1 — Prepare the manifest56A CSV with **one row per image** and a **patient/subject ID** column (`patient_id` / `subject_id` /57`case_id`), plus image and label path columns. The ID column is load-bearing: the split is done at the58patient level off this column.5960### Phase 2 — Generate the repo61```bash62python3 ${CLAUDE_SKILL_DIR}/scripts/scaffold.py \63 --manifest <manifest.csv> --task segmentation --out model_repo --seed 42 \64 --in-channels 1 --out-channels 165# --task = segmentation | classification | detection | synthesis | ssl | finetune66# (out-channels = num classes for classification/finetune, target channels for synthesis)67# fine-tuning a pretrained backbone (transfer learning) on collected clinical data:68python3 ${CLAUDE_SKILL_DIR}/scripts/scaffold.py \69 --manifest <manifest.csv> --task finetune --out model_repo --seed 42 \70 --out-channels <num_classes> --from-pretrained timm:resnet50.a1_in1k71# emits PRETRAINED.md (provenance) + a frozen→unfrozen train.py with discriminative LRs;72# record the exact pretrained source so the fine-tune is reproducible.73```74This writes `model_repo/` with `config.yaml`, `model.py` (the task's model — U-Net / CNN / Faster R-CNN75/ Pix2Pix / SimCLR encoder), `dataset.py` (reads the frozen split), `losses.py` (task-appropriate),76`train.py`, `evaluate.py`, `requirements.txt`,77`REPRODUCIBILITY.md`, `methods_stub.md`, and — the key artifact — `splits/split_assignment.csv` +78`splits/split_seed.txt`. The split is **patient-disjoint by construction** (a deterministic group split)79and the emitted code seeds every RNG, sets cuDNN deterministic, builds the training loader from the80**train split only**, and infers under `model.eval()` + `torch.no_grad()`.8182### Phase 3 — Verify the build (network-free)83```bash84# this skill's own training-hygiene gate85python3 ${CLAUDE_SKILL_DIR}/scripts/check_training_hygiene.py --repo model_repo --strict86# the split-leakage gate (proves patient disjointness) — owned by /model-validation87```88Route the emitted `splits/split_assignment.csv` to `/model-validation`89(`check_split_leakage.py --splits model_repo/splits/split_assignment.csv --strict`) for the90patient-disjointness proof, and (optionally, locally with torch installed)91`bash ${CLAUDE_SKILL_DIR}/scripts/scaffold_challenge/verify.sh` to smoke the forward pass.9293### Phase 4 — Plug in your data and train94Implement `dataset.py`'s `_load_image` / `_load_label` for your modality (DICOM / NIfTI / TIFF via95nibabel / pydicom / tifffile / TorchIO / MONAI transforms). For production, swap `model.py` for MONAI96`UNet` / `SegResNet` or an nnU-Net plan (see `${CLAUDE_SKILL_DIR}/references/training_guide.md`). For a97fine-tuning repo (`--task finetune`), fill `PRETRAINED.md` and set the freeze schedule / discriminative98learning rates (see `${CLAUDE_SKILL_DIR}/references/finetuning_guide.md`, which also covers MedSAM/SAM99adaptation and train-only diffusion augmentation). Run `python train.py` (best model selected on the100**val** split), then `python evaluate.py` (predictions on the **test** split, touched once).101102### Phase 5 — Validate, evaluate, publish103Hand off to `/model-validation` (validation-tier + comparator + metric-selection audit),104`/model-evaluation` + `/analyze-stats` (Dice + HD95/NSD with CIs), `/make-figures`, and `/write-paper`105(fill the `methods_stub.md` `[VERIFY]` placeholders) + `/check-reporting` (CLAIM 2024 / TRIPOD+AI). For106reproducibility-safe wiring of experiment tracking (W&B / MLflow), config / data / environment107versioning, and the MLOps reporting checklist, see `${CLAUDE_SKILL_DIR}/references/mlops_guide.md`108(a wiring + reporting reference — it points to the frameworks, it does not replace them).109110## Runnability — honest contract111The generated repo is **runnable**, but runnability is **not a CI guarantee**. The default gates prove112the network-free properties (the emitted split is patient-disjoint + seeded; the emitted training code113is hygienic) by parsing the produced artifacts — no torch is executed. A torch forward-pass smoke114(`build + forward shape + gradients flow + reproducible loss`) is a **self-skipping** tier in the115challenge `verify.sh` and a documented local command; it is never counted as CI coverage of116runnability.117118## Anti-Hallucination119120- **Never fabricate training or evaluation metrics.** The scaffold emits `[VERIFY]` placeholders;121 every number must come from the user's executed run and from `/model-evaluation` + `/analyze-stats`.122- **Never emit a split that is not patient-disjoint or not seed-locked.** The generator does this by123 construction; do not hand-edit the split table to introduce overlap or remove the seed.124- **Never claim the generated repo was trained or that it achieved a result** — it is a starting point125 the user runs.126- If a library API, default, or architecture detail is uncertain, flag `[VERIFY]` and ask rather than127 guessing.128129## Deterministic gates130- `scripts/scaffold.py` — the generator (stdlib + numpy; deterministic given manifest + seed).131- `scripts/check_training_hygiene.py` — AST linter: all RNGs seeded, cuDNN deterministic,132 `eval()` + `no_grad()` inference, no training on a non-train split, and (fine-tuning) a133 recorded pretrained-weight provenance when pretrained weights are loaded134 (`PRETRAINED_PROVENANCE_MISSING`).135- `scripts/scaffold_challenge/verify.sh` — the build → validate chain, network-free (torch tier136 self-skips).137138## Boundaries139140```141architecture-zoo (choose)142 └─ model-scaffold (this skill: generate the reproducible repo)143 ├─ check_training_hygiene.py (training-code hygiene)144 ├─ model-validation (split-leakage proof + validation design)145 ├─ model-evaluation -> analyze-stats (metrics + CIs)146 └─ write-paper + check-reporting (Methods stub -> compliant manuscript)147```