Medtech Model Evidence Export to MLflow
Purpose
Mirror an existing medical-inference result or evidence pack into MLflow after
the run and emit the export_result JSON contract. Keep the original evidence
pack as the source of truth. Training skills should add MLflow inside their
training loops instead.
Instructions
- Run
scripts/export_evidence_pack.py in the default dry-run mode.
- Inspect
params, metrics, artifact_plan, and mlflow.note.content.
- Choose
--mode local or --mode databricks only after checking the target.
- Keep
--artifact-policy metadata unless the target is approved for images.
- For
preview or all in a live mode, also pass
--confirm-medical-artifact-upload.
- Keep
--source-ref, --note, config filenames, and artifact filenames free
of patient or secret identifiers; always review the dry-run output first.
Hosts with a script helper can use
run_script("scripts/export_evidence_pack.py", args=["PACK_OR_RESULT", "--mode", "dry-run"]).
Available Scripts
| Script |
Purpose |
Arguments |
scripts/export_evidence_pack.py |
Export post-hoc inference evidence through MLflow. |
PACK_OR_RESULT --mode dry-run --artifact-policy metadata |
Prerequisites
- Python 3.10+.
mlflow>=2.10,<4 for local or databricks mode.
numpy>=1.24,<3 and nibabel>=4,<6 for NIfTI quality metrics and previews.
MLFLOW_TRACKING_URI may select a caller-managed tracking server.
- Databricks mode uses the caller's
DATABRICKS_HOST, DATABRICKS_TOKEN, or
configured Databricks profile. The declared network endpoint is
https://<caller-provided-mlflow-or-databricks-workspace>; Docker and GPU
are not required.
- Local mode may write the MLflow store under
<current-working-directory>/mlruns.
Examples
Preview the export without contacting MLflow:
python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \
runs/inference_pack --mode dry-run --artifact-policy metadata
Export a direct NV-Generate result with reproducibility metadata:
python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \
runs/nv-generate/result.json \
--mode local \
--experiment-name medical-ai-inference \
--config configs/chest_lung_tumor.json \
--seed 0 \
--source-ref git:61c4ec709b84cad468852243c48e250bec732074
Log downsampled slice previews, but not raw NIfTI files:
python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \
runs/nv-generate/result.json \
--mode databricks \
--experiment-name /Shared/medical-ai-inference \
--artifact-policy preview \
--confirm-medical-artifact-upload
--artifact-policy all additionally uploads discovered or explicitly supplied
NIfTI images and masks, subject to --max-artifact-mb. Use --image and
--mask when paths are not present in the result JSON.
The exporter logs:
- scalar run and quality metrics, including sampled HU mean/std/min/max for CT
(generic intensity statistics otherwise), a documented intensity-SNR
heuristic, mask foreground percentage, and mapped tumor volume percentage
when a tumor label mapping is available;
- generation parameters, model/checkpoint identity, RNG seed, and recipe hash;
- source config digest or
--source-ref, plus a prompt digest when present;
mlflow.note.content with a short human-readable run summary;
- a sanitized metadata bundle by default, optional PNG slice previews, and
raw image/mask artifacts only under the explicit
all policy.
Limitations
- This is post-hoc inference export, not live training-curve tracking.
- Global intensity SNR and downsampled volume statistics are engineering
checks, not image-quality or clinical-performance claims.
- Preview and raw artifacts may contain sensitive medical information. The
caller must approve the destination and data policy before upload.
- The exporter does not evaluate model quality, register models, or alter the
source evidence pack.
Troubleshooting
| Error |
Cause |
Fix |
| Evidence source not recognized |
No direct result JSON or pack manifest.json. |
Pass the result file, evidence-pack directory, or trusted-run root. |
| MLflow import fails |
Live mode lacks the declared package. |
Install mlflow>=2.10,<4 or use --mode dry-run. |
| Preview/all confirmation error |
A live image upload was not acknowledged. |
Review the destination, then pass --confirm-medical-artifact-upload. |
| Referenced image not found |
Result paths moved after inference. |
Pass current paths with --image and --mask. |
1---2name: medtech-model-evidence-export3description: Exports sanitized metadata, parameters, reproducibility details, quality metrics, and optional review artifacts from Medical AI inference runs or evidence packs to MLflow. Use after inference, including NV-Generate runs; not for live training tracking, model registration, or clinical use.4license: Apache-2.05allowed-tools: Bash6permissions: [env, file_read, file_write, network, shell]7metadata:8 source: https://skillmd.com/skills/nvidia/medtech-model-evidence-export9 author: 'NVIDIA MedTech <noreply@nvidia.com>'10---1112# Medtech Model Evidence Export to MLflow1314## Purpose1516Mirror an existing medical-inference result or evidence pack into MLflow after17the run and emit the `export_result` JSON contract. Keep the original evidence18pack as the source of truth. Training skills should add MLflow inside their19training loops instead.2021## Instructions22231. Run `scripts/export_evidence_pack.py` in the default `dry-run` mode.242. Inspect `params`, `metrics`, `artifact_plan`, and `mlflow.note.content`.253. Choose `--mode local` or `--mode databricks` only after checking the target.264. Keep `--artifact-policy metadata` unless the target is approved for images.275. For `preview` or `all` in a live mode, also pass28 `--confirm-medical-artifact-upload`.296. Keep `--source-ref`, `--note`, config filenames, and artifact filenames free30 of patient or secret identifiers; always review the dry-run output first.3132Hosts with a script helper can use33`run_script("scripts/export_evidence_pack.py", args=["PACK_OR_RESULT", "--mode", "dry-run"])`.3435## Available Scripts3637| Script | Purpose | Arguments |38|---|---|---|39| `scripts/export_evidence_pack.py` | Export post-hoc inference evidence through MLflow. | `PACK_OR_RESULT --mode dry-run --artifact-policy metadata` |4041## Prerequisites4243- Python 3.10+.44- `mlflow>=2.10,<4` for `local` or `databricks` mode.45- `numpy>=1.24,<3` and `nibabel>=4,<6` for NIfTI quality metrics and previews.46- `MLFLOW_TRACKING_URI` may select a caller-managed tracking server.47- Databricks mode uses the caller's `DATABRICKS_HOST`, `DATABRICKS_TOKEN`, or48 configured Databricks profile. The declared network endpoint is49 `https://<caller-provided-mlflow-or-databricks-workspace>`; Docker and GPU50 are not required.51- Local mode may write the MLflow store under52 `<current-working-directory>/mlruns`.5354## Examples5556Preview the export without contacting MLflow:5758```bash59python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \60 runs/inference_pack --mode dry-run --artifact-policy metadata61```6263Export a direct NV-Generate result with reproducibility metadata:6465```bash66python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \67 runs/nv-generate/result.json \68 --mode local \69 --experiment-name medical-ai-inference \70 --config configs/chest_lung_tumor.json \71 --seed 0 \72 --source-ref git:61c4ec709b84cad468852243c48e250bec73207473```7475Log downsampled slice previews, but not raw NIfTI files:7677```bash78python skills/medtech-model-evidence-export/scripts/export_evidence_pack.py \79 runs/nv-generate/result.json \80 --mode databricks \81 --experiment-name /Shared/medical-ai-inference \82 --artifact-policy preview \83 --confirm-medical-artifact-upload84```8586`--artifact-policy all` additionally uploads discovered or explicitly supplied87NIfTI images and masks, subject to `--max-artifact-mb`. Use `--image` and88`--mask` when paths are not present in the result JSON.8990The exporter logs:9192- scalar run and quality metrics, including sampled HU mean/std/min/max for CT93 (generic intensity statistics otherwise), a documented intensity-SNR94 heuristic, mask foreground percentage, and mapped tumor volume percentage95 when a tumor label mapping is available;96- generation parameters, model/checkpoint identity, RNG seed, and recipe hash;97- source config digest or `--source-ref`, plus a prompt digest when present;98- `mlflow.note.content` with a short human-readable run summary;99- a sanitized metadata bundle by default, optional PNG slice previews, and100 raw image/mask artifacts only under the explicit `all` policy.101102## Limitations103104- This is post-hoc inference export, not live training-curve tracking.105- Global intensity SNR and downsampled volume statistics are engineering106 checks, not image-quality or clinical-performance claims.107- Preview and raw artifacts may contain sensitive medical information. The108 caller must approve the destination and data policy before upload.109- The exporter does not evaluate model quality, register models, or alter the110 source evidence pack.111112## Troubleshooting113114| Error | Cause | Fix |115|---|---|---|116| Evidence source not recognized | No direct result JSON or pack `manifest.json`. | Pass the result file, evidence-pack directory, or trusted-run root. |117| MLflow import fails | Live mode lacks the declared package. | Install `mlflow>=2.10,<4` or use `--mode dry-run`. |118| Preview/all confirmation error | A live image upload was not acknowledged. | Review the destination, then pass `--confirm-medical-artifact-upload`. |119| Referenced image not found | Result paths moved after inference. | Pass current paths with `--image` and `--mask`. |