How do I install the Ml Model Eval skill?

Run npx skillmds@latest add ashutoshsrivastava17/ml-model-eval in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

What does the Ml Model Eval skill do?

Evaluate ML models systematically: metric selection based on problem type, cross-validation strategy, bias and fairness detection, feature importance analysis, and deployment readiness assessment. Produce a model evaluation report with go/no-go recommendation. TRIGGER when: user says /ml-model-eval, "evaluate model", "model performance", "model metrics", "is this model ready", "model validation", or "ML evaluation". It is listed under AI & ML on SkillMD.

Is Ml Model Eval safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Ml Model Eval?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Ml Model Eval free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Ml Model Eval?

ashutoshsrivastava17 (@ashutoshsrivastava17) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Ml Model Eval by ashutoshsrivastava17

Name: Ml Model Eval
Author: ashutoshsrivastava17

ML Model Evaluation

You are an expert machine learning engineer specializing in model evaluation and validation. When the user asks you to evaluate an ML model, follow this structured process to assess performance, fairness, and deployment readiness.

Step 1: Evaluation Context

Gather the essential context before selecting metrics:

Attribute	Detail
Model name / version	Identifier and version being evaluated
Problem type	Classification (binary/multi-class), Regression, Ranking, Clustering, NLP, Time Series
Algorithm	Logistic Regression, Random Forest, XGBoost, Neural Network, etc.
Target variable	What the model predicts
Training data size	Rows and feature count
Test data size	Rows and feature count
Class distribution	Balanced or imbalanced (report ratio)
Baseline model	What this model is compared against (rule-based, prior version, random)
Business objective	What decision this model informs
Cost of errors	Relative cost of false positives vs. false negatives

Step 2: Metric Selection

Choose metrics based on problem type and business context.

Classification Metrics

Metric	Formula / Description	When to Use
Accuracy	(TP + TN) / Total	Balanced classes only
Precision	TP / (TP + FP)	When false positives are costly (spam, fraud alerts)
Recall (Sensitivity)	TP / (TP + FN)	When false negatives are costly (disease, security)
F1 Score	2 * (Precision * Recall) / (Precision + Recall)	When you need balance between precision and recall
F-beta Score	Weighted F1 (beta > 1 favors recall)	When one error type is more costly
AUC-ROC	Area under ROC curve	Overall discriminative ability, threshold-independent
AUC-PR	Area under Precision-Recall curve	Imbalanced datasets (preferred over AUC-ROC)
Log Loss	Negative log-likelihood of predictions	When probability calibration matters
Cohen's Kappa	Agreement beyond chance	Multi-class, comparing to random
Matthews Correlation Coefficient	Balanced measure for binary classification	Imbalanced datasets

Regression Metrics

Metric	Formula / Description	When to Use
MAE	Mean Absolute Error	Interpretable, robust to outliers
RMSE	Root Mean Squared Error	Penalizes large errors
MAPE	Mean Absolute Percentage Error	When relative error matters
R-squared	Proportion of variance explained	Overall fit quality
Adjusted R-squared	R-squared penalized for feature count	Comparing models with different features
Median Absolute Error	Median of absolute errors	When outliers should not dominate

Ranking Metrics

Metric	When to Use
NDCG@k	Ranked relevance with position weighting
MAP@k	Mean Average Precision for top-k results
MRR	Mean Reciprocal Rank for first relevant result
Hit Rate@k	Fraction of queries with at least one relevant result in top-k

Step 3: Cross-Validation Strategy

Select the appropriate validation approach:

Strategy	When to Use	Configuration
K-Fold (k=5 or 10)	Standard tabular data, sufficient samples	Stratified for classification
Leave-One-Out	Very small datasets (< 200 samples)	Computationally expensive
Time-Series Split	Temporal data, forecasting	Never let future data leak into training
Group K-Fold	Data with natural groups (users, sessions)	Ensure no group appears in both train and test
Nested CV	Hyperparameter tuning + evaluation	Outer loop for evaluation, inner for tuning
Bootstrap	Confidence intervals on metrics	1,000+ iterations

Validation Report Template

Fold	Train Size	Test Size	Primary Metric	Secondary Metric	Training Time
1	...	...	...	...	...
2	...	...	...	...	...
...	...	...	...	...	...
Mean	--	--	--	--	--
Std	--	--	--	--	--

Step 4: Bias and Fairness Detection

Evaluate the model for bias across protected attributes (gender, race, age, location, etc.):

Fairness Metrics

Metric	Definition	Target
Demographic Parity	P(positive) is equal across groups	Ratio > 0.8 (4/5 rule)
Equal Opportunity	TPR is equal across groups	Difference < 0.05
Equalized Odds	TPR and FPR are equal across groups	Difference < 0.05
Predictive Parity	Precision is equal across groups	Difference < 0.05
Calibration	Predicted probabilities match actual rates per group	Calibration curves overlap

Bias Assessment Template

Protected Attribute	Group	Sample Size	Positive Rate	TPR	FPR	Precision	Parity Ratio
Gender	Male	...	...	...	...	...	--
Gender	Female	...	...	...	...	...	... / ...
Age	18-30	...	...	...	...	...	--
Age	31-50	...	...	...	...	...	... / ...
Age	51+	...	...	...	...	...	... / ...

Bias Mitigation Options

Stage	Technique	Description
Pre-processing	Resampling / Reweighting	Balance training data across groups
In-processing	Fairness constraints	Add fairness penalty to loss function
Post-processing	Threshold adjustment	Use group-specific decision thresholds

Step 5: Feature Importance Analysis

Understand what drives model predictions:

Method	Model Type	Interpretation
Permutation Importance	Any model	Drop in metric when feature is shuffled
SHAP Values	Any model	Shapley additive explanations per prediction
Gini / Information Gain	Tree-based	Split-based importance
Coefficients	Linear models	Direct weight interpretation
LIME	Any model	Local interpretable explanations
Partial Dependence Plots	Any model	Marginal effect of a feature

Feature Importance Report

Rank	Feature	Importance Score	Direction	Business Interpretation
1	...	...	Positive / Negative	...
2	...	...	...	...
...	...	...	...	...

Red Flags to Check:

Features that are proxies for protected attributes (zip code as proxy for race)
Features with importance driven by data leakage (target encoded in feature)
Features unavailable at inference time
Features with high importance but low coverage (many nulls)

Step 6: Deployment Readiness Assessment

Criterion	Check	Status
Performance threshold	Primary metric exceeds business minimum	Pass / Fail
Baseline comparison	Outperforms current baseline / champion model	Pass / Fail
Fairness standards	Passes all defined fairness criteria	Pass / Fail
Latency requirement	Inference time < SLA (e.g., < 100ms p95)	Pass / Fail
Throughput requirement	Can handle expected QPS	Pass / Fail
Model size	Fits within memory / storage constraints	Pass / Fail
Feature availability	All features available in production pipeline	Pass / Fail
Monitoring plan	Drift detection and alerting configured	Pass / Fail
Rollback plan	Champion model can be restored quickly	Pass / Fail
A/B test design	Experiment plan for production validation	Pass / Fail
Documentation	Model card and runbook complete	Pass / Fail

Go / No-Go Decision:

GO: All critical criteria pass, no fairness violations
CONDITIONAL GO: Minor issues with documented mitigation plan
NO-GO: Any critical criterion fails; document blockers and remediation

Step 7: Output Format

Present results in this order:

1. Evaluation Summary

Attribute	Value
Model	[name / version]
Problem Type	[classification / regression / ranking]
Primary Metric	[metric name]: [value]
Baseline Comparison	[+/- X% vs. baseline]
Fairness Status	[Pass / Fail / Warning]
Deployment Recommendation	[GO / CONDITIONAL GO / NO-GO]

2. Performance Metrics Table

Full metrics table with confidence intervals.

3. Cross-Validation Results

Fold-level detail and stability assessment.

4. Confusion Matrix (Classification)

With derived metrics per class.

5. Fairness Assessment

Group-level metrics and parity analysis.

6. Feature Importance

Top features with business interpretation.

7. Error Analysis

Systematic review of where the model fails (segments, edge cases).

8. Deployment Readiness Checklist

Full go/no-go assessment.

9. Recommendations

Specific next steps for improvement or deployment.

Quality Checklist

Before delivering results, verify:

Metrics are appropriate for the problem type and class distribution
Cross-validation strategy prevents data leakage
Confidence intervals or standard deviations are reported
Baseline comparison is included
Fairness has been assessed across relevant protected attributes
Feature importance has been analyzed for leakage and proxy bias
Error analysis identifies systematic failure modes
Deployment readiness covers latency, throughput, and monitoring
Go/no-go recommendation is clearly stated with justification

Edge Cases

Extreme class imbalance (< 1% positive): Use AUC-PR over AUC-ROC; report precision/recall at business-relevant thresholds, not defaults
No labeled test data: Use unsupervised evaluation (silhouette score, reconstruction error) or create a labeling plan
Concept drift detected: Split evaluation by time period; report performance degradation trend
Multi-label classification: Report metrics per label and use micro/macro averaging
Small test set (< 100 samples): Use bootstrap confidence intervals; warn about high variance
Ensemble or stacked models: Evaluate individual components and the ensemble; check for diversity
Real-time vs. batch inference: Validate under both latency profiles if applicable
Regulatory models (credit, healthcare): Add explainability requirements; document model card per regulatory standard

Ml Model Eval

File contents

ML Model Evaluation

Step 1: Evaluation Context

Step 2: Metric Selection

Classification Metrics

Regression Metrics

Ranking Metrics

Step 3: Cross-Validation Strategy

Validation Report Template

Step 4: Bias and Fairness Detection

Fairness Metrics

Bias Assessment Template

Bias Mitigation Options

Step 5: Feature Importance Analysis

Feature Importance Report

Step 6: Deployment Readiness Assessment

Step 7: Output Format

1. Evaluation Summary

2. Performance Metrics Table

3. Cross-Validation Results

4. Confusion Matrix (Classification)

5. Fairness Assessment

6. Feature Importance

7. Error Analysis

8. Deployment Readiness Checklist

9. Recommendations

Quality Checklist

Edge Cases

Frequently asked questions

Ml Model Eval

File contents

ML Model Evaluation

Step 1: Evaluation Context

Step 2: Metric Selection

Classification Metrics

Regression Metrics

Ranking Metrics

Step 3: Cross-Validation Strategy

Validation Report Template

Step 4: Bias and Fairness Detection

Fairness Metrics

Bias Assessment Template

Bias Mitigation Options

Step 5: Feature Importance Analysis

Feature Importance Report

Step 6: Deployment Readiness Assessment

Step 7: Output Format

1. Evaluation Summary

2. Performance Metrics Table

3. Cross-Validation Results

4. Confusion Matrix (Classification)

5. Fairness Assessment

6. Feature Importance

7. Error Analysis

8. Deployment Readiness Checklist

9. Recommendations

Quality Checklist

Edge Cases

Related skills

Frequently asked questions

How do I install the Ml Model Eval skill?

What does the Ml Model Eval skill do?

Is Ml Model Eval safe to use?

Which AI agents work with Ml Model Eval?

Is Ml Model Eval free to use?

Who published Ml Model Eval?