majiayu000 name: model-compare description: Compare 3D CAD models using boolean operations (IoU, Dice, precision/recall). Use when evaluating generated models against gold references, diffing CAD revisions, or computing similarity metrics for ML training. Triggers on: model diff, compare models, IoU, intersection over union, model similarity, CAD comparison, STEP diff, 3D evaluation, gold reference, generated model, precision recall 3D.
3D Model Comparison Tool
Compare CAD models using boolean operations to compute similarity metrics like IoU, Dice, precision, and recall. Useful for:
- Evaluating ML-generated models against gold references
- Comparing revisions of CAD designs
- Computing metrics for training 3D generative models
- Visualizing geometric differences
Quick Start
# Compare two STEP files
uvx --from build123d python scripts/model_diff.py reference.step generated.step
# JSON output for training pipelines
uvx --from build123d python scripts/model_diff.py ref.step gen.step --json --no-export
# Demo mode (no files needed)
uvx --from build123d python scripts/model_diff.py --demo
Supported Formats
| Format | Extension | Notes |
|---|---|---|
| STEP | .step, .stp |
Recommended - full CAD fidelity |
| BREP | .brep |
OpenCASCADE native format |
| STL | .stl |
Mesh format - may have boolean issues |
Output Metrics
Primary Metrics (for ML training)
| Metric | Range | Description |
|---|---|---|
| IoU (Jaccard) | 0-1 | ` |
| Dice (F1) | 0-1 | `2 |
| Precision | 0-1 | ` |
| Recall | 0-1 | ` |
Diagnostic Metrics
| Metric | Description |
|---|---|
volume_ratio |
B/A volume ratio (1.0 = same size) |
center_offset |
Distance between centers of mass |
bbox_iou |
Bounding box IoU (coarse alignment) |
size_ratio_x/y/z |
Per-axis scale comparison |
surface_ratio |
Surface area comparison |
Interpretation
The tool provides automatic interpretation:
- Over-generating: Low precision, high extra geometry
- Under-generating: Low recall, missing geometry
- Size issues: Volume ratio far from 1.0
- Position issues: Large center offset
CLI Options
usage: model_diff.py [-h] [-o OUTPUT_DIR] [--json] [--no-export] [--demo]
[reference] [generated]
positional arguments:
reference Reference/gold model file (STEP, BREP, or STL)
generated Generated/predicted model file to compare
options:
-o, --output-dir Output directory for GLB files (default: .)
--json Output only JSON metrics (for pipelines)
--no-export Skip exporting GLB visualization files
--demo Run with built-in demo models
Output Files
When --no-export is not set, produces GLB files for visualization:
| File | Description |
|---|---|
diff_reference.glb |
The reference model (A) |
diff_generated.glb |
The generated model (B) |
diff_missing.glb |
Geometry in A but not B (under-generation) |
diff_extra.glb |
Geometry in B but not A (over-generation) |
diff_common.glb |
Geometry in both (correct match) |
Example: Training Pipeline Integration
# Batch evaluation
for gen in outputs/*.step; do
uvx --from build123d python model_diff.py gold.step "$gen" --json --no-export
done | jq -s '{
avg_iou: (map(.iou) | add / length),
avg_precision: (map(.precision) | add / length),
avg_recall: (map(.recall) | add / length)
}'
Example: Loss Function
# In your training code, use metrics for loss:
loss = (
(1 - metrics['iou']) * 1.0 + # Primary shape match
abs(1 - metrics['volume_ratio']) * 0.5 + # Scale accuracy
metrics['center_offset'] * 0.1 # Position accuracy
)
How It Works
The tool uses boolean operations from OpenCASCADE (via build123d):
Missing = Reference - Generated (A - B)
Extra = Generated - Reference (B - A)
Common = Reference & Generated (A ∩ B)
Union = Reference + Generated (A ∪ B)
IoU = volume(Common) / volume(Union)
Dice = 2 * volume(Common) / (volume(A) + volume(B))
Precision = volume(Common) / volume(B)
Recall = volume(Common) / volume(A)
Sample Output
=================================================================
3D MODEL COMPARISON REPORT
Reference (A) vs Generated (B)
=================================================================
──────────────────────────── VOLUMES ────────────────────────────
Reference (A): 51,433.629
Generated (B): 45,904.426
Intersection (A∩B): 42,292.031
Missing (A-B): 9,141.598 (17.8% of A)
Extra (B-A): 3,612.395 (7.9% of B)
──────────────────────── PRIMARY METRICS ────────────────────────
IoU (Jaccard): 0.7683 (1.0 = identical)
Dice (F1): 0.8690 (1.0 = identical)
Precision: 0.9213 (correctness of B)
Recall: 0.8223 (coverage of A)
──────────────────────── INTERPRETATION ─────────────────────────
△ Partial match (IoU > 50%)
→ Under-generating: 17.8% of A is missing
→ Undersized by 10.8%
=================================================================