# Trident > Process whole-slide pathology images (WSIs) with TRIDENT: tissue segmentation, patch coordinate extraction, and patch/slide feature (embedding) extraction with foundation models (UNI, CONCH, Virchow, Gemma, Titan, GigaPath, etc.). Use when the user works with WSIs (.svs/.tiff/.ndpi/.mrxs/.czi/.dcm), mentions TRIDENT, run_batch_of_slides / run_single_slide, tissue segmentation, patching, or extracting pathology embeddings for downstream ML. - Skill: `mahmoodlab/trident` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add mahmoodlab/trident` - Raw SKILL.md: https://api.skillmd.com/api/skills/mahmoodlab/trident/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: mahmoodlab (https://skillmd.com/u/mahmoodlab) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/mahmoodlab/trident --- # TRIDENT — whole-slide image processing TRIDENT runs a 3-stage pipeline over whole-slide images (WSIs): ``` tissue segmentation → patch coordinates → patch / slide embeddings (--task seg) (--task coords) (--task feat) ``` Outputs are written under a single `--job_dir` and are **resumable** — re-running the same command skips finished work. **Tasks run one stage only — they do NOT auto-run prerequisites.** Pick `--task`: - `--task all` — runs seg → coords → feat in one go (the usual choice). Requires a `--patch_encoder` (or `--slide_encoder`); it always produces features. - `--task seg` / `--task coords` / `--task feat` — run *only* that stage. `coords` needs segmentation already done in `--job_dir`; `feat` needs coords already done (or supply `--coords_dir`). Running `--task coords` on a fresh `--job_dir` produces nothing (it skips with `geojson_not_found`). - **Seg + coords but no features?** There is no single flag — run two commands: `--task seg` then `--task coords` on the same `--job_dir`. For the full CLI flag list, the complete encoder tables, output layout, and the Python API, read **[reference.md](reference.md)**. Keep this file for the workflow and decisions. ## Setup (do this once, verify before big jobs) ```bash pip install -e . # core; add ".[patch-encoders]" ".[slide-encoders]" ".[full]" as needed trident-doctor --profile base # preflight; use --profile --check-gated for model access ``` - **Versions** — Python 3.10/3.11 (`>=3.10,<3.13`), `timm>=0.9.16,<2`, `transformers>=4.51,<5`. Stay on transformers 4.x: v5 removes `transformers.onnx`, which Hibou-L's remote code imports. - **flash-attn** — needed only by `gigapath`, `gigapath-flash` and `prism2`. Blackwell GPUs (sm_100/sm_120) require `>=2.7.3`; earlier releases ship no kernels for those cards. PyPI has only an sdist, so install a prebuilt wheel from [releases](https://github.com/Dao-AILab/flash-attention/releases). - If `trident-doctor` isn't on PATH (install-dependent), preflight instead with `python -c "import trident; from trident.patch_encoder_models import encoder_factory; encoder_factory('uni_v1')"`. - Most encoders download from HuggingFace; gated models (UNI, CONCH, Virchow, …) need an approved HF account and `huggingface-cli login`. A load failure usually means missing access or a missing optional install — read the error, it names the fix. - `gemma4-e4b`/`gemma4-26b` are the one exception to the transformers range: they need `>=5`, which breaks `hibou_l` and `titan`. Use a separate env; don't upgrade a shared one. - A **slide encoder** that errors on load with something like `all_tied_weights_keys` (not a gating/timm error) is a `transformers` 5.x incompatibility (e.g. TITAN) — pin `transformers` 4.x, or if you can't change the env, set `transformers.modeling_utils.PreTrainedModel.all_tied_weights_keys = {}` before loading (for the batch CLI's spawned workers, put that in a `sitecustomize.py` on `PYTHONPATH`). ## The one command users want first ```bash python run_batch_of_slides.py --task all \ --wsi_dir ./wsis --job_dir ./trident_processed \ --patch_encoder uni_v1 --mag 20 --patch_size 256 --gpus 0 ``` Segments tissue, extracts patches at 20× / 256px, and writes UNI embeddings. For a single slide (cautious first run), use `run_single_slide.py --slide_path ./wsis/x.svs ...` with the same flags. From another project, the CLI equivalents are `trident batch -- ...` / `trident single -- ...`. ## Decisions you must get right **1. The encoder dictates `--patch_size` and `--mag` — do not pick them freely.** Each model was trained at a specific resolution; mismatched values give garbage features. Always copy the pair from the encoder table in [reference.md](reference.md). Common ones: | Encoder | use | |---|---| | `uni_v1` (1024-d) | `--patch_size 256 --mag 20` | | `uni_v2` (1536-d) | `--patch_size 256 --mag 20` | | `conch_v15` (768-d, default) | `--patch_size 512 --mag 20` | | `virchow` / `virchow2` (2560-d) | `--patch_size 224 --mag 20` | | `virchow2-cls` (1280-d, CLS only — what PRISM2 consumes) | `--patch_size 224 --mag 20` | | `gigapath` (1536-d) / `gigapath-flash` (384-d) | `--patch_size 256 --mag 20` | | `ctranspath` (768-d) | `--patch_size 256 --mag 10` | **2. Patch vs slide embeddings.** `--patch_encoder X` → one embedding per patch (`features_X/`, shape `(n_patches, dim)`). `--slide_encoder Y` → one embedding per slide (`slide_features_Y/`, shape `(dim,)`); it auto-runs the correct patch encoder first. Pass the slide encoder's required patch_size/mag (from the slide-encoder table). **Still pass `--task all`** (or `feat`) with `--slide_encoder` — on its own, the default `--task seg` only segments and you get no embeddings. A slide-encoder run also writes the intermediate patch features (`features_/`) alongside `slide_features_/`. ("UNI" = `uni_v1`; "UNI2"/"UNI2-h" = `uni_v2`.) Each slide encoder is hard-wired to one patch encoder, so the intermediate folder name follows *that* encoder, not the slide encoder. Two pairs are easy to trip over: `prism` uses `virchow` (2560-d) while `prism2` uses `virchow2-cls` (1280-d, class token only — **not** `virchow2`, which is 2560-d), and `gigapath-flash` uses its own 384-d `gigapath-flash` patch encoder. Features from the two Virchow2 flavours are therefore never interchangeable; they live in separate `features_virchow2/` vs `features_virchow2-cls/` folders by design. **3. Segmenter.** Default `--segmenter hest` (a model — runs on GPU). `grandqc` = fast H&E. `otsu` = classical, **CPU-only** — on a machine with no GPU you must pass `--segmenter otsu` explicitly (the default `hest` expects a GPU). For segmentation, `--gpus -1` and `otsu` go together. - If segmentation misses tissue, lower `--seg_conf_thresh` (default 0.5 → try 0.4) to retain more. - Optional clean-up: `--remove_penmarks` (gentle) or `--remove_artifacts` (aggressive: folds, blur, stains, OOF…). - ⚠️ `--remove_artifacts` keeps only "normal tissue" and can erase an entire slide whose pyramid reads soft (e.g. some MIRAX/`.mrxs` slides → flagged out-of-focus). If a slide ends up with **no patches/features**, check the slide's `wsi_states/` JSON for reason `artifact_removal_emptied_tissue` and re-run that slide with `--remove_penmarks` (it trades away fold/blur/OOF removal to keep the tissue). The two flags are not combinable. **4. Compute.** `--gpus 0 1 2` shards pending slides across GPUs; `--gpus -1` forces CPU. Tune throughput with `--batch_size` / `--feat_batch_size` / `--seg_batch_size` (raise on big GPUs). For slow/network storage, stage slides locally with `--wsi_cache /local/ssd --cache_batch_size 32`. **5. mpp / custom inputs.** To process a **subset**, or to supply **micron-per-pixel** for slides that lack embedded metadata, use `--custom_list_of_wsis list.csv`. The CSV has a `wsi` column (filename incl. extension, relative to `--wsi_dir`) and, when slides have no embedded mpp, a required `mpp` column with the value for each row: ```csv wsi,mpp slide_001.tiff,0.5 slide_002.tiff,0.25 ``` Use the CSV `mpp` column to *supply* a value; use `--custom_mpp_keys` only when the value *is* in the slide metadata under a non-standard key. `--mag` in batch mode accepts floats (e.g. `1.25`). **6. Reuse existing patch coordinates** (e.g. legacy CLAM `*_fp/`): skip seg+patching and point feat at them — `--task feat --coords_dir ./extracted_mag20x_patch256_fp ...` with the encoder's required patch_size/mag (must match the coords' resolution). `--wsi_dir` is still required (features read pixels from the WSIs). ## Outputs (under `--job_dir`) ``` thumbnails/.jpg slide thumbnail contours/.jpg thumbnail + tissue contour overlay contours_geojson/.geojson tissue polygons (editable in QuPath) _config_segmentation.json args used for the seg run _logs_segmentation.txt per-slide seg status x_px_px_overlap/ e.g. 20x_256px_0px_overlap/ patches/_patches.h5 patch coords (dataset `coords`, (n,2), + patch attrs) visualization/.jpg patch-grid overlay patch_images//*.png patch image crops — only with --dump_patches features_/.h5 patch embeddings: datasets `features` (n,dim) + `coords` slide_features_/.h5 slide embedding `features` (dim,) — only with --slide_encoder _config_coords.json / _config_feats_.json / _config_slide_features_.json _logs_coords.txt / _logs_feats_.txt / _logs_slide_features_.txt summary.md human-readable run report (one section per run) runs/.json machine-readable run manifest wsi_states/__.json per-slide tasks, attempts, errors, resume reason .lock transient in-flight guard (cleared by --clear_dead_locks) ``` A feature `.h5` is **self-contained** (it stores `coords` alongside `features`). The patch/feature subdir is named only from `mag/patch/overlap`, so changing those creates a *separate* folder rather than reusing one. Full per-artifact detail + h5 attrs: see [reference.md](reference.md). To inspect results: open an `.h5` with `h5py` and read `coords` / `features`; check `summary.md` or `wsi_states/` to see what succeeded, was skipped, or errored, and why. To also export the actual **patch images** (not just coordinates), add `--dump_patches` to a `coords`/`all` run — writes PNGs (or `--dump_patches_format jpg`) to `/patch_images//`, capped by `--dump_patches_max` (see reference.md). ## Workflow for the agent 1. Confirm the goal: which **encoder** (drives patch_size/mag), patch vs slide embeddings, GPU availability. 2. Verify env (`trident-doctor`, or the `encoder_factory('uni_v1')` import fallback if it's not on PATH), and HF access for gated encoders. 3. Build the command from the recipe above; pull the exact patch_size/mag from [reference.md](reference.md). Prefer `run_single_slide.py` to validate on one slide first. 4. Run; then **verify**: confirm the expected `features_/` / `slide_features_/` files exist with sensible shapes, and scan `wsi_states/` for skips/errors. Report any slide that produced no output and the reason. 5. Re-run on the **same `--job_dir` with identical `--mag`/`--patch_size`/`--overlap`/encoder** to resume (changing them starts a new output dir instead of resuming). Finished slides are skipped; errored/unfinished ones are retried. Add `--clear_dead_locks` only if a previous run was killed and left stale `.lock` files. ## Common pitfalls - Mismatched `--patch_size`/`--mag` for the chosen encoder → meaningless features. - **`--overlap` must be `< --patch_size`.** `--overlap >= --patch_size` makes the patch step ≤ 0 and **hangs forever** (no error). Overlap is absolute pixels (e.g. `128` = 50% of a 256px patch). - Re-patching with a changed `--min_tissue_proportion` (or seg settings) on an existing `--job_dir` has **no effect** — the coords folder is keyed only by `mag/patch/overlap`, so the old coords are reused. Use a fresh `--job_dir` to re-patch. - `--patch_encoder_img_size` and `--patch_encoder_ckpt_path` apply only to `--patch_encoder`; they are silently ignored when `--slide_encoder` is set. - `--task coords`/`feat` on a fresh `--job_dir` → silently skips (no prior stage); use `--task all`, or run the stages in order. - `--slide_encoder` without `--task all`/`feat` → only segmentation runs, no embeddings. - No-GPU machine without `--segmenter otsu` → default `hest` tries to use a GPU. - `timm` outside `>=0.9.16,<2` → cryptic model-build errors that can look like a model *load* failure. - Gated HF model without access → load failure (request access + `huggingface-cli login`). - `FlashAttention only supports Ampere GPUs or newer` from `gigapath`/`gigapath-flash`/`prism2` on a Blackwell GPU → flash-attn <2.7.3 has no kernels for that arch (see Setup). Confirm with `trident-doctor --profile slide-encoders`, which checks flash-attn against the live GPU. - `gemma4-*` asserting on transformers → it needs v5 (see Setup). - Empty output after `--remove_artifacts` → see Decision 3. - Changing `--mag`/`--patch_size`/`--overlap` on a rerun → new output folder instead of a resume. - Wrong reader auto-detected → force it with `--reader_type {openslide,image,cucim,sdpc,omezarr,czi}`.