UitbreidenOS Evaluation Framework Builder
When to activate
When a user needs to design, implement, or validate evaluation frameworks for ML/AI models — including metric selection, benchmark design, test harness setup, and quality assurance strategies for model outputs.
When NOT to use
- For general model training or fine-tuning (use model-training or fine-tuning-orchestrator)
- For deployment health monitoring (use observability or monitoring skills)
- For data preparation or preprocessing workflows (use data-engineering skills)
- For prompt optimization without evaluation context (use prompt-engineering skills)
Instructions
TBD — This skill covers:
- Evaluation metric design and selection
- Benchmark dataset construction
- Test harness architecture and automation
- Quality thresholds and regression testing
- A/B testing framework setup
- Quantitative and qualitative evaluation strategies
Example
TBD — Example evaluation framework for a semantic search model (embedding quality, latency, recall metrics, benchmark construction).