kermt-embed
Extract per-molecule embeddings from any encoder-bearing KERMT checkpoint.
The skill is the workflow orchestrator: validate ckpt, validate CSV, clean
SMILES, launch the runner blocking, return the per-readout .npy files.
Skill and runtime paths
Set SKILL_DIR to the absolute path of this installed skill directory. Export
KERMT_REPO as the absolute path to the KERMT checkout used for model
execution. The bundled container helper mounts that checkout at
/workspace and this skill at /skill (read-only). Commands inside
the container use /skill/scripts/; defaults are bundled in config/.
See Released models for checkpoint bundle requirements.
Downloads and local outputs
The optional released-model branch reads config/released_model.json for the
Hugging Face repository, pinned revision, and filenames. The bundled
scripts/fetch_released_model.py downloads the model bundle over HTTPS into
the host directory the user selects. Public models work without credentials;
if HF_TOKEN is set, the container helper forwards it for Hugging Face
authentication. Prepared data, logs, and workflow results go into the chosen
run directory.
Hardware requirements
- GPUs: 1 (single-GPU).
- VRAM: ≥ 4 GB for the default
batch_size 64.
- Disk: depends on output size — roughly a few MB per 1k molecules at
hidden 800 per readout, so ~10–20 MB per 1k molecules across the 4
readouts. Plus a small canonical_smiles.npy + validity.npy per run.
- Driver / CUDA: any host supporting CUDA 12.6.
Inputs
Required:
--csv <path> — SMILES CSV. First column is smiles; other columns
are ignored (no targets needed).
Checkpoint (optional — defaults to the released model if omitted):
--ckpt <path> — any encoder-bearing checkpoint. Grover_base, cmim,
hybrid, and finetuned ckpts are all accepted. The validator only refuses
ckpts with no encoder. If omitted, the skill offers to download the
released pretrained hybrid model nvidia/NV-KERMT-70M-v2 and embed with
it — see "Resolve & validate the checkpoint" (workflow step 3).
--pretrained-release — explicit opt-in to use the released model without
the interactive prompt (for non-interactive / agent runs). Mutually
exclusive with --ckpt.
--model-dir <dir> — where to save the downloaded bundle (default
$KERMT_REPO/models/NV-KERMT-70M-v2/). An already-complete bundle there is
reused, not re-downloaded.
Optional:
--batch-size N — override the configured default (64).
--gpus 0 — single GPU id (default 0).
--from-prepare <dir> — skip the prepare step and reuse an existing
prepare_data.json in <dir>.
Workflow
Let $KERMT_REPO be the path to your kermt repo checkout.
Pre-flight: container + system probe.
"$SKILL_DIR/scripts/kermt_container.sh" check_system
Compute run directory.
RUN_DIR=$KERMT_REPO/runs/embed_$(date -u +%Y-%m-%dT%H-%M-%SZ)
Resolve & validate the checkpoint.
Resolve — only if --ckpt was omitted. Default to the released
pretrained hybrid model nvidia/NV-KERMT-70M-v2:
Validate the resolved (or user-provided) ckpt:
"$SKILL_DIR/scripts/kermt_container.sh" run --ckpt <user-ckpt> -- \
"python /skill/scripts/check_checkpoint.py --mode embed --ckpt /ckpt"
Parse JSON. Abort on ok: false. The validator only refuses encoder-less
ckpts (rare).
Validate the data.
"$SKILL_DIR/scripts/kermt_container.sh" run --data <user-csv> -- \
"python /skill/scripts/check_data.py --mode embed --csv /data/<basename>"
Prepare the data (clean-only — no features step).
"$SKILL_DIR/scripts/kermt_container.sh" run --data <user-csv> --run-dir $RUN_DIR -- \
"python /skill/scripts/prepare_data.py --mode embed \\
--csv /data/<basename> --out /runs/data"
Outputs land at $RUN_DIR/data/prepare_data.json with a single clean_csv
path. task/extract_embeddings.py featurizes from SMILES on the fly.
Launch the runner (blocking).
"$SKILL_DIR/scripts/kermt_container.sh" run \\
--ckpt <user-ckpt> --run-dir $RUN_DIR -- \\
"python /skill/scripts/run_extract_embeddings.py \\
--ckpt /ckpt \\
--prepare-manifest /runs/data/prepare_data.json \\
--out /runs \\
[--gpus 0 --batch-size N]"
Report to the user.
- Embeddings directory:
$RUN_DIR/out/
atom_from_atom.npy, bond_from_atom.npy,
atom_from_bond.npy, bond_from_bond.npy (the 4 standard readouts;
each shape (N_rows, hidden_size))
metadata.pkl — pickle of a dict containing canonical_smiles
(RDKit-canonicalized SMILES per row), valid (boolean per-row: did
RDKit parse it), plus other run metadata.
- Manifest:
$RUN_DIR/run.json
- Log:
$RUN_DIR/logs/embed.log
Hard rules
- Never download the released model without consent. When
--ckpt is
omitted, download nvidia/NV-KERMT-70M-v2 only after an explicit user "yes"
or an explicit --pretrained-release flag. --ckpt and
--pretrained-release are mutually exclusive.
- Never modify the user's ckpt. The runner reads-only via
task/extract_embeddings.py's --checkpoint <path> flag.
- Arch comes from the ckpt. No
--hidden-size flag etc. on this runner;
task/extract_embeddings.py reads arch from the ckpt's saved_args.
Common errors
prepare_data manifest is missing required output 'clean_csv' → prepare
ran with --skip-clean but no source CSV given. Re-run prepare without it.
--gpus '0,1' is single-GPU only → pass a single id.
Replayability
$(jq -r .cmd_replay $RUN_DIR/run.json)
If ok_to_replay: false (dirty kermt repo worktree at launch time), pin
the commit via repo.commit and git checkout it first.
1---2name: kermt-embed3description: Extract per-molecule embeddings from any encoder-bearing KERMT checkpoint. Use a local checkpoint or optionally download a pinned Hugging Face model bundle using HF_TOKEN if configured. Run containerized embedding extraction and write model bundles, per-readout .npy embeddings, canonical SMILES, and validity arrays to user-selected host directories.4license: Apache-2.05compatibility: Requires docker, nvidia-container-toolkit, and a CUDA-capable NVIDIA GPU. Designed for Claude Code, Codex, and Nemotron.6metadata:7 source: https://skillmd.com/skills/nvidia/kermt-embed8 owner: evax@nvidia.com9 classification: workflow-skill10 risk_tier: skill11# Line/token budget: targets ~150 lines / ~1800 tokens — within the12# 500-line / 5000-token cap for skill files.13---1415# kermt-embed1617Extract per-molecule embeddings from any encoder-bearing KERMT checkpoint.18The skill is the workflow orchestrator: validate ckpt, validate CSV, clean19SMILES, launch the runner blocking, return the per-readout `.npy` files.2021## Skill and runtime paths2223Set `SKILL_DIR` to the absolute path of this installed skill directory. Export24`KERMT_REPO` as the absolute path to the KERMT checkout used for model25execution. The bundled container helper mounts that checkout at26`/workspace` and this skill at `/skill` (read-only). Commands inside27the container use `/skill/scripts/`; defaults are bundled in `config/`.28See [Released models](references/released-models.md) for checkpoint bundle requirements.2930## Downloads and local outputs3132The optional released-model branch reads `config/released_model.json` for the33Hugging Face repository, pinned revision, and filenames. The bundled34`scripts/fetch_released_model.py` downloads the model bundle over HTTPS into35the host directory the user selects. Public models work without credentials;36if `HF_TOKEN` is set, the container helper forwards it for Hugging Face37authentication. Prepared data, logs, and workflow results go into the chosen38run directory.3940## Hardware requirements4142- **GPUs**: 1 (single-GPU).43- **VRAM**: ≥ 4 GB for the default `batch_size 64`.44- **Disk**: depends on output size — roughly a few MB per 1k molecules at45 `hidden 800` per readout, so ~10–20 MB per 1k molecules across the 446 readouts. Plus a small `canonical_smiles.npy` + `validity.npy` per run.47- **Driver / CUDA**: any host supporting CUDA 12.6.4849## Inputs5051Required:5253- `--csv <path>` — SMILES CSV. First column is `smiles`; other columns54 are ignored (no targets needed).5556Checkpoint (optional — defaults to the released model if omitted):5758- `--ckpt <path>` — any encoder-bearing checkpoint. Grover_base, cmim,59 hybrid, and finetuned ckpts are all accepted. The validator only refuses60 ckpts with no encoder. **If omitted**, the skill offers to download the61 released pretrained hybrid model **nvidia/NV-KERMT-70M-v2** and embed with62 it — see "Resolve & validate the checkpoint" (workflow step 3).63- `--pretrained-release` — explicit opt-in to use the released model without64 the interactive prompt (for non-interactive / agent runs). Mutually65 exclusive with `--ckpt`.66- `--model-dir <dir>` — where to save the downloaded bundle (default67 `$KERMT_REPO/models/NV-KERMT-70M-v2/`). An already-complete bundle there is68 reused, not re-downloaded.6970Optional:7172- `--batch-size N` — override the configured default (64).73- `--gpus 0` — single GPU id (default 0).74- `--from-prepare <dir>` — skip the prepare step and reuse an existing75 `prepare_data.json` in `<dir>`.7677## Workflow7879Let `$KERMT_REPO` be the path to your kermt repo checkout.80811. **Pre-flight: container + system probe.**82 ```83 "$SKILL_DIR/scripts/kermt_container.sh" check_system84 ```85862. **Compute run directory.**87 ```88 RUN_DIR=$KERMT_REPO/runs/embed_$(date -u +%Y-%m-%dT%H-%M-%SZ)89 ```90913. **Resolve & validate the checkpoint.**9293 **Resolve — only if `--ckpt` was omitted.** Default to the released94 pretrained hybrid model **nvidia/NV-KERMT-70M-v2**:95 - **Consent gate.** Unless `--pretrained-release` was passed, ask the user:96 "No checkpoint given — download the released model nvidia/NV-KERMT-70M-v297 (NVIDIA Open Model License, https://huggingface.co/nvidia/NV-KERMT-70M-v2)98 and embed with it? [y/N]". **Never download without an explicit yes** (or99 `--pretrained-release`). If both `--ckpt` and `--pretrained-release` are100 given, abort — they conflict.101 - **Save location.** Default `$KERMT_REPO/models/NV-KERMT-70M-v2/`; honor102 `--model-dir <dir>` if given. An already-complete bundle is reused.103 - **Download** (foreground; ~282 MB on first fetch):104 ```105 "$SKILL_DIR/scripts/kermt_container.sh" run --model-dir <save-dir> -- \106 "python /skill/scripts/fetch_released_model.py --out /model"107 ```108 Parse the JSON; abort on `ok: false` (surface `errors`). On success set109 `<user-ckpt> = <save-dir>/kermt_contrastive_v2.0.pt`.110111 **Validate** the resolved (or user-provided) ckpt:112 ```113 "$SKILL_DIR/scripts/kermt_container.sh" run --ckpt <user-ckpt> -- \114 "python /skill/scripts/check_checkpoint.py --mode embed --ckpt /ckpt"115 ```116 Parse JSON. Abort on `ok: false`. The validator only refuses encoder-less117 ckpts (rare).1181194. **Validate the data.**120 ```121 "$SKILL_DIR/scripts/kermt_container.sh" run --data <user-csv> -- \122 "python /skill/scripts/check_data.py --mode embed --csv /data/<basename>"123 ```1241255. **Prepare the data** (clean-only — no features step).126 ```127 "$SKILL_DIR/scripts/kermt_container.sh" run --data <user-csv> --run-dir $RUN_DIR -- \128 "python /skill/scripts/prepare_data.py --mode embed \\129 --csv /data/<basename> --out /runs/data"130 ```131 Outputs land at `$RUN_DIR/data/prepare_data.json` with a single `clean_csv`132 path. `task/extract_embeddings.py` featurizes from SMILES on the fly.1331346. **Launch the runner (blocking).**135 ```136 "$SKILL_DIR/scripts/kermt_container.sh" run \\137 --ckpt <user-ckpt> --run-dir $RUN_DIR -- \\138 "python /skill/scripts/run_extract_embeddings.py \\139 --ckpt /ckpt \\140 --prepare-manifest /runs/data/prepare_data.json \\141 --out /runs \\142 [--gpus 0 --batch-size N]"143 ```1441457. **Report to the user.**146 - Embeddings directory: `$RUN_DIR/out/`147 - `atom_from_atom.npy`, `bond_from_atom.npy`,148 `atom_from_bond.npy`, `bond_from_bond.npy` (the 4 standard readouts;149 each shape `(N_rows, hidden_size)`)150 - `metadata.pkl` — pickle of a dict containing `canonical_smiles`151 (RDKit-canonicalized SMILES per row), `valid` (boolean per-row: did152 RDKit parse it), plus other run metadata.153 - Manifest: `$RUN_DIR/run.json`154 - Log: `$RUN_DIR/logs/embed.log`155156## Hard rules157158- **Never download the released model without consent.** When `--ckpt` is159 omitted, download `nvidia/NV-KERMT-70M-v2` only after an explicit user "yes"160 or an explicit `--pretrained-release` flag. `--ckpt` and161 `--pretrained-release` are mutually exclusive.162- **Never modify the user's ckpt.** The runner reads-only via163 `task/extract_embeddings.py`'s `--checkpoint <path>` flag.164- **Arch comes from the ckpt.** No `--hidden-size` flag etc. on this runner;165 `task/extract_embeddings.py` reads arch from the ckpt's saved_args.166167## Common errors168169- `prepare_data manifest is missing required output 'clean_csv'` → prepare170 ran with `--skip-clean` but no source CSV given. Re-run prepare without it.171- `--gpus '0,1' is single-GPU only` → pass a single id.172173## Replayability174175```bash176$(jq -r .cmd_replay $RUN_DIR/run.json)177```178179If `ok_to_replay: false` (dirty kermt repo worktree at launch time), pin180the commit via `repo.commit` and `git checkout` it first.