MR Synthesis Quality Verifier
Purpose
- Used for deterministic second-pass review of
nv-generate-mrandnv-generate-mr-brainevidence packs. - Re-reads generated image artifacts and checks source-pack success, upstream entrypoint identity, model inventory, modality/version identity, shape and spacing consistency, finite nonconstant nonnegative voxel values, aggregate-flag honesty, runtime identity, and scope disclosure.
- Manifest I/O: inputs are
nv_generate_mr_evidence_pack; outputs aremr_synthesis_quality_report.
Instructions
- Use this verifier only on evidence packs produced by
skills/nv-generate-mrorskills/nv-generate-mr-brain. - Run it through
eval_engine/run.pywhen producing verifier evidence. - The verifier entrypoint is
scripts/grade.py; do not replace it with prompt-only review. - Treat a pass as an engineering floor only, not as validation of anatomical realism, diagnostic utility, or training-data suitability.
Available Scripts
| Script | Purpose | Arguments |
|---|---|---|
scripts/grade.py |
Primary verifier entrypoint declared by skill_manifest.yaml. |
EVIDENCE_PACK_DIR |
Prerequisites
- Runtime requirements: Python packages listed in
runtime.side_effects.pip_packages. - The input directory must contain
manifest.json,validation_summary.json, andoutput.jsonfrom a supported MR generation evidence pack. - Referenced generated image artifacts must be available for NIfTI header/data checks.
Limitations
- Does not validate anatomical realism, radiologic image quality, diagnosis, regulatory claims, or production training-data suitability.
- Basic numeric checks catch empty/corrupt outputs, not subtle model failures.
- Does not re-run the upstream diffusion model.
Troubleshooting
| Error | Cause | Fix |
|---|---|---|
target_skill_matches fails |
The fixture is not an nv-generate-mr or nv-generate-mr-brain evidence pack. |
Re-run the verifier against a supported source pack. |
image_artifacts_readable fails |
A generated NIfTI path is missing, moved, or unreadable. | Regenerate or restore the source evidence pack artifacts. |
image_values_nonconstant fails |
The generated image is constant or nearly empty. | Inspect upstream logs and regenerate evidence after fixing runtime/input issues. |
aggregate_flags_match_recomputed fails |
Source output flags disagree with verifier recomputation. | Fix the wrapper summary logic and regenerate the source pack. |
Example
python eval_engine/run.py verifiers/mr_synthesis_quality_v1 \
--fixture runs/nv_generate_mr_demo \
--out runs/mr_synthesis_quality
python verifiers/mr_synthesis_quality_v1/scripts/grade.py \
runs/nv_generate_mr_demo