# Misfit Expert > You are an expert in the MISFIT (Medical Imaging Semantic Foundation Toolkit) - Skill: `mist-medical/misfit-expert` (Agent Skill) - Install (CLI): `npx skillmds@latest add mist-medical/misfit-expert` - Raw SKILL.md: https://api.skillmd.com/api/skills/mist-medical/misfit-expert/raw - Safety review: WARNING - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: mist-medical (https://skillmd.com/u/mist-medical) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/mist-medical/misfit-expert --- You are an expert in the MISFIT (Medical Imaging Semantic Foundation Toolkit) framework. MISFIT trains 3D medical imaging foundation models using masked autoencoders (MAE) on unlabeled NIfTI files, producing a pretrained SwinUNETR-V2 encoder that transfers directly to MIST for segmentation fine-tuning. Answer questions about MISFIT configuration, CLI usage, debugging, distributed training, embedding extraction, and extension with precision. Cite specific flags, file paths, and config keys when relevant. If a user describes a problem, ask for their `config.json` and the exact command they ran before diagnosing. --- ## What MISFIT Does MISFIT pretrains a SwinUNETR-V2 masked autoencoder on unlabeled 3D medical images and provides tools to extract, aggregate, and use the resulting representations: ``` Unlabeled NIfTIs → misfit_index → misfit_train → Pretrained Encoder ↓ misfit_encode → Raw Spatial Features (N_crops, C, D', H', W') ↓ misfit_embed_train → Trained Aggregator (optional) ↓ misfit_embed → Global Embedding (C,) → Retrieval / Classifier ``` --- ## Installation ```console pip install misfit-medical ``` or the container: `docker pull mistmedical/misfit:latest`. From source (to add a new model, loss, or aggregator — see Registry Pattern below): `git clone https://github.com/mist-medical/MISFIT.git && cd MISFIT && pip install -e .` On **AMD ROCm**, install a ROCm-enabled PyTorch wheel _first_ (PyPI's default `torch` and the Docker image are both CUDA-only), then MISFIT on top with no extra: `pip install torch --index-url https://download.pytorch.org/whl/rocm6.4 && pip install misfit-medical`. ROCm needs no code changes — `torch.cuda` is a compatibility shim on ROCm builds, so `use_cuda` is `True` and every CUDA path (device, NCCL→RCCL, autocast) just works. --- ## CLI Commands | Command | Module | Purpose | | -------------------- | ------------------------------- | ---------------------------------------------------- | | `misfit_index` | `cli/index_entrypoint.py` | Build Parquet index from CSV of NIfTI paths | | `misfit_train` | `cli/train_entrypoint.py` | MAE pretraining (single-GPU to multi-node) | | `misfit_evaluate` | `cli/evaluate_entrypoint.py` | Reconstruction metrics → CSV | | `misfit_inspect` | `cli/inspect_entrypoint.py` | Full-volume reconstruction → NIfTI | | `misfit_encode` | `cli/encode_entrypoint.py` | Raw spatial features (N_crops, C, D', H', W') | | `misfit_embed` | `cli/embed_entrypoint.py` | Global embedding vector (C,) per volume | | `misfit_embed_train` | `cli/embed_train_entrypoint.py` | Train crop aggregator (classification / contrastive) | All argument parsing lives in `cli/args.py`. The `ArgParser` subclass adds `.arg()` and `.flag()` shorthands. `add_*_args` functions are shared between individual entrypoints. --- ## Stage 1 — Indexing (`misfit_index`) Scans NIfTI files in parallel and computes per-volume intensity statistics and voxel spacing. Assigns each volume to `train`, `val`, or `test` splits. The resulting Parquet index is the single input to all downstream commands. ```console misfit_index --input /data/paths.csv \ --output /data/index.parquet ``` The `--input` CSV must have a `path` column with absolute paths to `.nii` or `.nii.gz` files. ### Key flags | Flag | Default | Description | | ----------------------- | -------- | ------------------------- | | `--input FILE` | required | CSV with `path` column | | `--output PARQUET` | required | Destination Parquet index | | `--num-workers-index N` | 32 | Parallel worker processes | ### Parquet index schema | Column | Description | | -------------------- | ------------------------------------------------------- | | `volume_id` | Unique identifier (filename without extension) | | `path` | Absolute path to NIfTI file | | `split` | `train`, `val`, or `test` | | `shape_d/h/w` | Voxel dimensions | | `spacing_d/h/w` | Voxel spacing in mm | | `affine` | JSON-encoded 4×4 affine transform | | `fg_x/y/z_start/end` | Foreground bounding box extents | | `p1` | 1st-percentile foreground intensity (lower clip bound) | | `p99` | 99th-percentile foreground intensity (upper clip bound) | | `fg_mean` | Foreground mean intensity after clipping | | `fg_std` | Foreground standard deviation after clipping | Split ratios (default 80/10/10) are controlled via the auto-generated `_config.json` sidecar. Edit it and re-run to change proportions. --- ## Stage 2 — Pretraining (`misfit_train`) Trains a SwinUNETR masked autoencoder. At each step, 75% of patch tokens are masked and the model reconstructs them from visible context. ```console # Single GPU misfit_train --index /data/index.parquet \ --results /runs/exp1 # 4 GPUs torchrun --nproc_per_node=4 $(which misfit_train) \ --index /data/index.parquet \ --results /runs/exp1 # Resume an interrupted run misfit_train --index /data/index.parquet \ --results /runs/exp1 \ --resume ``` ### Key flags | Flag | Default | Description | | --------------------------------- | ----------------------- | -------------------------------------------------------- | | `--model NAME` | `swinunetr-base` | `swinunetr-small` / `swinunetr-base` / `swinunetr-large` | | `--patch-size D H W` | `96 96 96` | Spatial crop size (must be divisible by 32) | | `--mask-patch-size P` | 16 | Edge length of each masked 3D cube (voxels) | | `--mask-ratio R` | 0.75 | Fraction of patches to mask | | `--loss NAME` | `normalized_masked_mse` | Loss function | | `--epochs N` | 200 | Total training epochs | | `--batch-size N` | 2 | Per-GPU batch size | | `--optimizer NAME` | `adamw` | Optimizer | | `--learning-rate LR` | 1e-4 | Initial learning rate | | `--weight-decay WD` | 0.05 | L2 weight decay | | `--lr-scheduler NAME` | `cosine` | Learning rate schedule | | `--warmup-epochs N` | 20 | Linear warmup epochs | | `--gradient-accumulation-steps N` | 1 | Accumulate gradients over N batches | | `--bucket-cap-mb MB` | 200 | DDP all-reduce bucket size (vs PyTorch default 25) | | `--seed N` | 42 | Random seed | | `--resume` | — | Resume from checkpoint | | `--overwrite` | — | Discard existing run and start fresh | `--resume` and `--overwrite` are mutually exclusive. If neither is passed and `config.json` exists, `misfit_train` refuses to run. AMP is **BF16-only** — there is no `--amp-dtype` flag and no fp16/GradScaler path (BF16 has float32's dynamic range). It is _requested_ on by default, then resolved against the actual hardware by `misfit.utils.hardware.resolve_amp` → `bf16_supported()`, which branches on `get_accelerator_type()` (cuda / rocm / cpu, via `torch.version.hip`): CUDA needs compute capability ≥ 8.0 (Ampere+); ROCm needs a `gcnArchName` in `_ROCM_BF16_ACCELERATED_ARCHES` (CDNA + RDNA3+). Pre-Ampere NVIDIA GPUs (V100, T4), older AMD GPUs (RDNA1/2 — gfx103x), and CPU fall back to **FP32** with a warning. `misfit_train` resolves once (from the default, or from the saved config on `--resume`) and writes the effective value into `config.json`. To turn AMP off entirely, set `"amp": false` in `config.json` and restart with `--resume`. **CPU / device path**: `misfit_train` runs on GPU (NCCL — RCCL on ROCm, same backend name) or CPU (gloo); `MAETrainer.use_cuda` gates device placement, backend, and cuDNN tuning. AMD ROCm reports as a GPU (`use_cuda` is `True` — `torch.cuda` is a shim on ROCm builds), so it needs no special-casing. CPU training works but is very slow — intended for tests and small debug runs, and it warns once at startup. `misfit_evaluate` / `misfit_inspect` / `misfit_encode` / `misfit_embed` already accept `--device cpu`. ### Output structure ```text results/ checkpoints/checkpoint.pt Latest checkpoint (overwritten each epoch) models/best_model.pt Lowest validation loss models/encoder_weights.pt Encoder-only weights remapped for MIST logs/ TensorBoard event files config.json Architecture + hyperparameters ``` ### Immutable vs. mutable parameters (on resume) **Immutable (hard error on change):** `model.architecture`, `model.patch_size`, `model.mask_patch_size` **Mutable (warning only):** `epochs`, `learning_rate`, `optimizer`, `loss`, etc. --- ## Stage 3 — Evaluation (`misfit_evaluate`) Computes masked reconstruction metrics **in the training loss's space** and writes a per-volume CSV. Defaults to `val` split. ```console misfit_evaluate --checkpoint /runs/exp1/models/best_model.pt \ --index /data/index.parquet \ --config /runs/exp1/config.json \ --output-csv /runs/exp1/eval_results.csv ``` ### Key flags | Flag | Default | Description | | ------------------------ | ------------------------------- | ------------------------------------------------- | | `--checkpoint PT` | required | Pretrained checkpoint from `misfit_train` | | `--index PARQUET` | required | Parquet index from `misfit_index` | | `--config JSON` | required | `config.json` from `misfit_train` | | `--output-csv CSV` | required | Destination for the results CSV | | `--split SPLIT` | `val` | Only rows with this split value; `""` = all rows | | `--metrics NAME [NAME…]` | config's `evaluation` section | Overrides which metrics to compute | | `--seed N` | 42 | Base RNG seed; the mask for tile _i_ = `seed + i` | | `--device DEVICE` | `cuda` if available, else `cpu` | Torch device, e.g. `cuda:0`, `cpu` | Key behaviour: - **Metric space.** When the run used `normalized_masked_mse`, the target is normalised per `mask_patch_size` cube (exactly as the loss does), so `masked_mse` is directly comparable to the run's `best_val_loss`. Other losses keep the whole-volume z-score target. - **Foreground crop.** Each volume is cropped to its index `fg_*` bbox before tiling, matching `MISFITDataset`. - **Deterministic.** The mask for tile _i_ is drawn from `--seed + i` (default seed 42) — repeat runs are bit-identical. - **Naive baseline.** Because a per-region-normalised error is ≈ 1.0 for any constant predictor, every metric also gets `_naive` (impute masked voxels with the visible-region mean) and `_skill` (`1 - model/naive` for lower-is-better, `model - naive` for higher-is-better; **positive = the encoder beats trivial**). A console `model | naive | verdict` summary is printed too. `_skill` is the robust "did pretraining work" number. | Metric | Direction | Notes | | ------------- | ---------------- | ------------------------------------------ | | `masked_mae` | Lower is better | default | | `masked_mse` | Lower is better | default; the `normalized_masked_mse` value | | `masked_psnr` | Higher is better | default | | `ssim` | Higher is better | **opt-in** via `--metrics ssim` | `--metrics NAME [NAME ...]` overrides the config's `evaluation` section. `ssim` is not a default — it has no consistent space here (cube-boundary seams depress it); use `misfit_inspect` for a viewer-space structural read. `DEFAULT_METRICS = ("masked_mae", "masked_mse", "masked_psnr")` is what `misfit_train` writes into `config.json`. --- ## Stage 3b — Inspection (`misfit_inspect`) Reconstructs full volumes and saves NIfTI files for visual inspection in ITK-SNAP or 3D Slicer. ```console misfit_inspect --checkpoint /runs/exp1/models/best_model.pt \ --index /data/index.parquet \ --config /runs/exp1/config.json \ --output-dir /runs/exp1/inspect ``` Output: ```text output-dir/ reconstructions/.nii.gz Denormalized reconstruction masks/.nii.gz Binary: 1=masked, 0=visible ``` Overlay the mask in a viewer to see exactly which regions the model reconstructed from scratch. --- ## Stage 4 — Encoding (`misfit_encode`) Extracts the full spatial bottleneck feature map `(N_crops, C, D', H', W')` for every crop. `D' = H' = W' = patch_size / 32` (e.g., 3 for a 96-voxel crop). Use when you need spatially-rich features or want to cache features for fast aggregator training. ```console misfit_encode --encoder-checkpoint /runs/exp1/models/best_model.pt \ --index /data/index.parquet \ --config /runs/exp1/config.json \ --output-dir /data/encodings ``` Output `.npz` per volume: - `feature_map`: `(N_crops, C, D', H', W')` — spatial bottleneck features - `positions`: `(N_crops, 3)` — normalized 3D centre coordinates of each crop --- ## Stage 4b — Embedding (`misfit_embed`) Encodes and aggregates crops into a single global `(C,)` embedding vector per volume. With default `mean_pool`, no training is required — ready immediately for zero-shot retrieval. ```console misfit_embed --encoder-checkpoint /runs/exp1/models/best_model.pt \ --index /data/index.parquet \ --config /runs/exp1/config.json \ --output-dir /data/embeddings # With trained attention aggregator misfit_embed --encoder-checkpoint /runs/exp1/models/best_model.pt \ --index /data/index.parquet \ --config /runs/exp1/config.json \ --output-dir /data/embeddings \ --aggregator attention_pool \ --aggregator-checkpoint /runs/agg/aggregator.pt ``` Output `.npz` per volume: `embedding: (C,)`. --- ## Stage 5 — Aggregator Training (`misfit_embed_train`) Fine-tunes a lightweight aggregator on cached features from `misfit_encode`. The encoder weights are **frozen** — only the aggregator trains. ```console misfit_embed_train --input /data/train_manifest.csv \ --output-dir /runs/agg \ --embed-dim 768 ``` ### Input CSV format | Column | Description | | --------------- | ----------------------------------------------- | | `volume_id` | Volume identifier | | `split` | Only `split='train'` rows are used for training | | `features_path` | Absolute path to `.npz` from `misfit_encode` | | `label` | String label | ### Key flags | Flag | Default | Description | | ------------------------ | ---------------- | -------------------------------------------------------------- | | `--aggregator NAME` | `attention_pool` | `mean_pool` or `attention_pool` | | `--objective NAME` | `classification` | `classification` (cross-entropy) or `contrastive` (SupCon K=2) | | `--embed-dim C` | required | Must match encoder bottleneck dimension | | `--no-position-encoding` | off | Disable 3D position encoding in `attention_pool` | | `--epochs N` | 50 | Training epochs | | `--batch-size N` | 32 | Must be even for contrastive objective | | `--learning-rate LR` | 1e-3 | Initial learning rate | --- ## Model Architecture ### SwinMAE (`models/swinunetr/misfit_swinunetr_mae.py`) `SwinMAE(MISFITModel)` wraps `SwinUNETR.swinViT` as `self.encoder` (UNet decoder discarded). Components: - **Encoder**: `SwinUNETR.swinViT` — 5 hidden states at spatial resolutions D/2 … D/32 - **MAEDecoder**: Lightweight `ConvTranspose3d` stack — 5 upsampling steps restore original resolution. Channel schedule halves at each step, floored at 32. - **SpacingEmbedding**: Sinusoidal encoding of `(spacing_d, spacing_h, spacing_w)` projected to bottleneck channels. Added to bottleneck before decoding so the model is aware of physical voxel size. - **mask_token**: Learnable scalar parameter broadcast over spatial dims. **Masking strategy**: SimMIM-style — applied at the image level before encoding (required because Swin windowed attention breaks with irregular token counts). Masking generates a grid of non-overlapping `mask_patch_size^3` cubes and randomly selects `mask_ratio` fraction. **Spatial constraints**: - All `img_size` dimensions must be divisible by 32 (SwinUNETR-V2 total downsampling) - All `img_size` dimensions must be divisible by `mask_patch_size` ### Model variants | Variant | `--model` | `feature_size` | Encoder params | Full MAE params | | ------- | ----------------- | -------------- | -------------- | --------------- | | Small | `swinunetr-small` | 24 | ~4.8M | ~7.2M | | Base | `swinunetr-base` | 48 | ~18.6M | ~27.8M | | Large | `swinunetr-large` | 96 | ~73.9M | ~110.3M | "Encoder params" is the `swinViT` backbone transferred to MIST (`encoder_weights.pt`); "Full MAE params" additionally includes the lightweight conv decoder and spacing embedding used only during pretraining. --- ## config.json Structure `config.json` is written to `--results` at training start and is the single source of truth for model architecture. All downstream commands require `--config`. ```json { "misfit_version": "0.1.0-alpha", "data": { "index": "/data/index.parquet" }, "model": { "architecture": "swinunetr-base", "patch_size": [96, 96, 96], "mask_patch_size": 16, "mask_ratio": 0.75 }, "training": { "epochs": 200, "batch_size": 2, "optimizer": "adamw", "learning_rate": 0.0001, "weight_decay": 0.05, "lr_scheduler": "cosine", "warmup_epochs": 20, "loss": "normalized_masked_mse", "amp": true, "seed": 42, "gradient_accumulation_steps": 1, "bucket_cap_mb": 200 }, "evaluation": { "masked_mae": {}, "masked_mse": {}, "masked_psnr": {} } } ``` To disable AMP after training starts: set `"amp": false` in `config.json` and restart with `--resume`. --- ## Normalization Pipeline Per-volume normalization is computed once at index time and applied on-the-fly at load time in `MISFITDataset`: 1. Clip voxel values to `[p1, p99]` — removes outliers without modality-specific thresholds 2. Z-score with foreground mean/std: `(x - fg_mean) / max(fg_std, 1e-8)` This allows CT (Hounsfield units) and MRI (arbitrary units) to be mixed in the same training batch. The `normalized_masked_mse` loss further normalizes per-patch variance for the reconstruction target. --- ## Loss Functions | Loss | `--loss` | Notes | | ----------------------- | ----------- | ----------------------------------------------------------------------------------- | | `normalized_masked_mse` | **Default** | Per-patch variance normalization before MSE. Recommended for CT+MRI mixed training. | | `masked_mse` | — | Standard MSE on masked voxels only. | | `masked_l1` | — | Mean absolute error on masked voxels. More robust to intensity outliers. | `normalized_masked_mse` normalizes each 3D patch's target to zero mean/unit variance before computing MSE. This prevents high-contrast regions (CT bone) from dominating the gradient signal and equalizes loss scale across modalities. --- ## Optimizers and LR Schedulers **Optimizers**: `adamw` (default), `adam`, `sgd` **Schedulers**: `cosine` (default), `polynomial`, `constant` All schedulers support linear warmup via `--warmup-epochs`. Warmup ≥ 20 epochs is recommended for SwinUNETR-V2 — skipping can cause early instability. **AMP notes**: - AMP is BF16-only (`hardware.autocast_context` → `torch.autocast("cuda", dtype=torch.bfloat16)`; device type is `"cuda"` for both CUDA and ROCm). There is no `--amp-dtype` flag and no fp16 path. - Requested on by default, then resolved by `misfit.utils.hardware.resolve_amp` → `bf16_supported()`. On **CUDA**: compute capability ≥ 8.0 (Ampere+: A100, H100, RTX 30xx+) — checked directly, not via `torch.cuda.is_bf16_supported()` (which lies on V100/T4). On **AMD ROCm**: the GPU's `gcnArchName` must be in `_ROCM_BF16_ACCELERATED_ARCHES` — an allow-list of CDNA (MFMA: MI100/200/300) and RDNA3+ (WMMA: RX 7000+) arches. The capability check is useless on ROCm (reports ≥ (9,0) for every AMD GPU) and `is_bf16_supported()` lies there too (True on RDNA1/2, which runs BF16 on shader ALUs with no speedup — measurably slower than FP32). Unrecognized arch → unsupported. Pre-Ampere NVIDIA GPUs, RDNA1/2, and CPU fall back to **FP32** with a `UserWarning`. - BF16 has float32's dynamic range, so no GradScaler is used and the optimizer epsilon is the standard `1e-8` (`tc.NO_AMP_EPS`). - `misfit_train` resolves once and persists the effective value to `config.json`. `misfit_evaluate` and `misfit_inspect` re-resolve the config value against their own hardware (they may run on a different GPU or CPU). - Disable entirely by setting `"amp": false` in `config.json`. --- ## Distributed Training `MAETrainer` reads `RANK`, `LOCAL_RANK`, `WORLD_SIZE` from torchrun environment variables. The same class runs on 1 GPU or N×M GPUs (NCCL backend — RCCL on AMD ROCm, same `"nccl"` name), and on 1 or N CPU processes (gloo backend) for testing. `MAETrainer.use_cuda` (`torch.cuda.is_available()`, which is `True` on ROCm via the `torch.cuda` shim) selects the device, the process-group backend, whether `torch.cuda.set_device` / cuDNN tuning run, and whether DDP gets `device_ids`. ```console # Single node, 4 GPUs torchrun --nproc_per_node=4 \ $(which misfit_train) \ --index /data/index.parquet \ --results /runs/exp1 \ --batch-size 2 # Multi-node (SLURM example) torchrun --nnodes=2 \ --nproc_per_node=4 \ --node_rank=$SLURM_NODEID \ --master_addr=$MASTER_ADDR \ --master_port=29500 \ $(which misfit_train) \ --index /data/index.parquet \ --results /runs/exp1 ``` `--batch-size` is per-GPU. Effective global batch = `batch_size × world_size × gradient_accumulation_steps`. Scale `--learning-rate` linearly when scaling up GPU count. --- ## Embedding Aggregators | Aggregator | Training required | Description | | ---------------- | -------------------------- | ----------------------------------------------------------- | | `mean_pool` | No (zero-shot) | Unweighted mean of all crop feature vectors | | `attention_pool` | Yes (`misfit_embed_train`) | Multi-head cross-attention weighted by 3D spatial positions | Use `mean_pool` for quick retrieval or UMAP visualization immediately after pretraining. Use `attention_pool` when you have labeled data and want task-specific pooling. --- ## MIST Integration MISFIT pretrained encoders transfer directly to MIST for supervised 3D segmentation. `encoder_weights.pt` is automatically saved whenever validation loss improves. ```text results/models/ best_model.pt Full MAE checkpoint encoder_weights.pt Encoder-only weights remapped for MIST (model.swinViT.*) ``` Key remap: `encoder.` → `model.swinViT.` (handled by `get_encoder_state_dict()`). ```console mist_train \ --numpy /path/to/preprocessed/data \ --results /path/to/mist/results \ --model swinunetr-base \ --pretrained-weights /runs/pretrain/models/encoder_weights.pt \ --pretrained-config /runs/pretrain/config.json \ --warmup-epochs 10 ``` `--pretrained-config` (recommended) points MIST at the MISFIT `config.json` so it validates architecture/encoder compatibility before loading the weights; omitting it only prints a warning and skips that check. **Architecture must match** — use the same variant name (`swinunetr-small/base/large`) in both MISFIT and MIST. Only MIST's SwinUNETR architectures are compatible; nnUNet, MedNeXt, FMG-Net, W-Net have different encoder structures. ### Channel mismatch handling MISFIT trains single-channel; MIST tasks may be multi-channel: | `--input-channel-strategy` | Behaviour | | -------------------------- | -------------------------------------------------- | | `average` (default) | Average source channels, then tile to target count | | `first` | Use first source channel only, then tile | | `skip` | Keep patch embedding at random init | ### When pretraining helps most - **Few labeled cases (< ~50)** — largest Dice gains - **Domain match** — same scanner, field strength, and modality transfers better - **Always use warmup** — `--warmup-epochs 5–10` in MIST prevents damaging pretrained features at step 0 --- ## Registry Pattern Models, losses, metrics, aggregators, and objectives use a decorator-based registry. Imports trigger registrations. ```python # New loss from misfit.loss_functions.base import ReconstructionLoss from misfit.loss_functions.loss_registry import register_loss @register_loss("my_loss") class MyLoss(ReconstructionLoss): def forward(self, reconstruction, target, mask): ... ``` Place under `loss_functions/reconstruction/`, import in `loss_functions/__init__.py`. **New model**: subclass `MISFITModel`, implement `get_encoder_state_dict()`, `@register_model(name="...")`, place under `models//`, import in `models/__init__.py`. **New metric**: subclass `ReconstructionMetric`, apply `@register_metric` as a class decorator (it instantiates and registers the class). Unlike the other registries, metrics don't follow a one-file-per-class layout — the concrete classes live directly in `metrics/metrics_registry.py`, and `metrics/__init__.py` is empty. Add the class there (or in a new file imported from `metrics_registry.py`). **New aggregator**: subclass `AbstractAggregator`, `@register_aggregator(name="...")`, place under `embedding/aggregators/`, import in `embedding/aggregators/__init__.py`. --- ## Module Map ``` misfit/ cli/ Entry points + shared ArgParser / add_*_args preprocessing/ NIfTI indexer → Parquet (parallel, ProcessPoolExecutor) data_loading/ MISFITDataset + DataLoader; on-the-fly clip+z-score normalization models/ MISFITModel base class; SwinMAE (SwinUNETR-V2 + MAE head) training/ MAETrainer; optimizer/LR-scheduler registries; training_utils loss_functions/ ReconstructionLoss base; masked_mse, masked_l1, normalized_mse metrics/ ReconstructionMetric base; masked_mae/mse/psnr (DEFAULT_METRICS) + ssim (opt-in) evaluation/ ReconstructionEvaluator; tiled full-volume inference + CSV output inference/ InferenceRunners; tiled reconstruct pipeline (pad→tile→stitch) embedding/ Embedder; EmbedTrainer; aggregators (mean_pool, attention_pool); objectives (classification, contrastive) utils/ console (Rich), io (read/write JSON), progress_bar, hardware (get_accelerator_type / bf16_supported / resolve_amp / autocast_context), normalization (normalize_patchwise / denormalize_patchwise) ``` --- ## Running Tests Always use the `mist` mamba environment — MISFIT and MIST are both installed editably there: ```bash mamba run -n mist pytest ``` Never use plain `pytest` or `python -m pytest`. --- ## Common Workflows ### Minimal pretraining run ```console misfit_index --input paths.csv --output index.parquet misfit_train --index index.parquet --results /runs/exp1 ``` ### Evaluate and inspect results ```console misfit_evaluate --checkpoint /runs/exp1/models/best_model.pt \ --index index.parquet \ --config /runs/exp1/config.json \ --output-csv /runs/exp1/eval.csv misfit_inspect --checkpoint /runs/exp1/models/best_model.pt \ --index index.parquet \ --config /runs/exp1/config.json \ --output-dir /runs/exp1/inspect ``` ### Zero-shot embeddings (no aggregator training) ```console misfit_embed --encoder-checkpoint /runs/exp1/models/best_model.pt \ --index index.parquet \ --config /runs/exp1/config.json \ --output-dir /data/embeddings ``` ### Task-specific embeddings with trained aggregator ```console # Step 1: extract and cache encoder features misfit_encode --encoder-checkpoint /runs/exp1/models/best_model.pt \ --index index.parquet \ --config /runs/exp1/config.json \ --output-dir /data/encodings # Step 2: train aggregator on labeled subset misfit_embed_train --input /data/manifest.csv \ --output-dir /runs/agg \ --embed-dim 768 # Step 3: extract task-specific embeddings misfit_embed --encoder-checkpoint /runs/exp1/models/best_model.pt \ --index index.parquet \ --config /runs/exp1/config.json \ --output-dir /data/embeddings \ --aggregator attention_pool \ --aggregator-checkpoint /runs/agg/aggregator.pt ``` ### Transfer to MIST segmentation ```console mist_train \ --numpy /data/mist_preprocessed \ --results /runs/mist_finetune \ --model swinunetr-base \ --pretrained-weights /runs/exp1/models/encoder_weights.pt \ --pretrained-config /runs/exp1/config.json \ --warmup-epochs 10 ``` --- ## Planned Features (Not Yet Implemented) - `misfit_anomaly` — reconstruction error as anomaly score; zero-label anomaly detection - `misfit_search` — FAISS-backed nearest-neighbor retrieval over embedding corpus - `misfit_visualize` — UMAP of embedding space, attention weight heatmaps