Evaluate
Comprehensive quality evaluation for any AI-generated artifact. Produces its report as a visualization.
How It Works
┌──────────────────────────────────────────────┐
│ │
│ Phase 1: SPEC GENERATION │
│ Analyze the artifact type │
│ Generate tailored evaluation criteria │
│ Define scoring dimensions + weights │
│ Set quality gates │
│ │ │
│ ▼ │
│ Phase 2: EVALUATION │
│ Run automated checks (when possible) │
│ Visual/manual inspection │
│ Score each dimension with evidence │
│ Identify systemic vs local issues │
│ │ │
│ ▼ │
│ Phase 3: REPORT (via /visualize) │
│ Generate a beautiful HTML eval report │
│ Scores, charts, screenshots, fix list │
│ Radar chart of dimensions │
│ Before/after tracking │
│ │
└──────────────────────────────────────────────┘
Phase 1: Spec Generation
For any artifact, generate evaluation specs by analyzing:
1. Identify Artifact Type
- HTML Visualization → visual design, interactivity, technical, content, shareability
- Code/Project → correctness, readability, architecture, test coverage, performance
- Document/Report → clarity, structure, accuracy, completeness, tone
- Conversation/Agent → helpfulness, accuracy, tone, efficiency, safety
- Slide Deck → all visualization dims + narrative flow, persuasion, pacing
- Dashboard → data accuracy, information density, scannability, actionability
- Custom → derive dimensions from the skill's SKILL.md and stated goals
2. Generate Dimensions
For each artifact type, produce 6-10 evaluation dimensions. Each dimension needs:
- Name — short, clear label
- Description — what this dimension measures
- Weight — percentage (all weights sum to 100%)
- Scoring anchors — what does a 10, 8, 6, 4 look like?
- Automated checks — any programmatic tests (if applicable)
- Deductions — specific issues and their point costs
3. Set Quality Gates
Define gates based on the artifact's purpose:
| Gate |
Criteria |
Meaning |
| 🚀 EXCEPTIONAL |
Overall ≥ 9.5, all ≥ 9 |
Best-in-class. Share everywhere. |
| ✅ SHIP |
Overall ≥ 9.0, all ≥ 8 |
Production-ready. |
| ⚠️ ACCEPTABLE |
Overall ≥ 8.0, all ≥ 7 |
Usable but not impressive. |
| 🔧 NEEDS WORK |
Overall ≥ 7.0 or any < 7 |
Fix before releasing. |
| ❌ FAIL |
Overall < 7.0 or any < 5 |
Major rework. |
4. Output Spec Document
Write the spec to eval-spec-[artifact-name].md for reference and reuse.
Phase 2: Evaluation
For HTML Visualizations
Open in browser at 3 viewports (1280×720, 768×1024, 375×667).
Automated audit (run in browser console):
(function() {
const audit = {};
const style = [...document.querySelectorAll('style')].map(s => s.textContent).join(' ');
const html = document.documentElement.outerHTML;
// Structure
audit.hasDoctype = /^<!doctype html>/i.test(html);
audit.hasLangAttr = !!document.documentElement.lang;
audit.hasCharset = !!document.querySelector('meta[charset]');
audit.hasViewport = !!document.querySelector('meta[name="viewport"]');
audit.hasTitle = document.title.length > 0;
// Menu system
audit.menuExists = !!document.querySelector('.viz-menu');
audit.menuHasTheme = !!html.match(/cycleTheme|themeLabel/i);
audit.menuHasDownload = !!html.match(/htmlToImage|html-to-image/i);
audit.menuHasPrint = !!html.match(/window\.print/i);
// Theme system
audit.hasCSSVars = !!style.match(/--bg\s*:/);
audit.hasDarkTheme = !!style.match(/(\.theme-dark|:root)[\s\S]*?--bg/);
audit.hasLightTheme = !!style.match(/\.theme-light/);
audit.themePersistedToStorage = !!html.match(/localStorage.*theme/i);
// Typography
audit.hasInterFont = !!html.match(/fonts\.googleapis.*Inter|font-family.*Inter/i);
audit.hasFontFallback = !!style.match(/-apple-system|system-ui/);
audit.bodyFontSize = parseFloat(getComputedStyle(document.body).fontSize);
audit.bodyFontOK = audit.bodyFontSize >= 14;
// Layout
audit.usesFlexOrGrid = !!(style.match(/display\s*:\s*(flex|grid)/));
audit.hasMaxWidth = !!style.match(/max-width/);
audit.hasResponsiveBreakpoints = !!style.match(/@media.*max-width|@media.*min-width|sm:|md:|lg:/);
// Print & Accessibility
audit.hasPrintStyles = !!style.match(/@media\s*print/);
audit.hasPrintColorAdjust = !!style.match(/print-color-adjust/);
audit.hasReducedMotion = !!style.match(/prefers-reduced-motion/);
audit.hasAriaLabels = !!html.match(/aria-label/);
audit.hasSemanticHTML = !!html.match(/<(header|main|nav|section|article|footer)/);
// Animations
audit.hasKeyframes = !!style.match(/@keyframes/);
audit.hasTransitions = !!style.match(/transition\s*:/);
// Performance
audit.fileSizeKB = Math.round(new Blob([html]).size / 1024);
audit.fileSizeOK = audit.fileSizeKB < 200;
audit.noExternalImages = document.querySelectorAll('img[src^="http"]').length === 0;
audit.htmlToImageLoaded = typeof htmlToImage !== 'undefined';
// Summary
const bools = Object.entries(audit).filter(([k,v]) => typeof v === 'boolean');
const passed = bools.filter(([k,v]) => v).length;
audit._passed = passed;
audit._total = bools.length;
audit._percent = Math.round(passed / bools.length * 100);
audit._failures = bools.filter(([k,v]) => !v).map(([k]) => k);
console.table(audit);
return audit;
})();
Visual scoring — 8 dimensions for visualizations:
| # |
Dimension |
Weight |
10 = |
6 = |
| D1 |
First Impression |
15% |
Apple keynote quality |
Generic template feel |
| D2 |
Typography |
15% |
Perfect hierarchy, Inter font, fluid sizing |
All same size, no hierarchy |
| D3 |
Color & Contrast |
10% |
Harmonious, WCAG AA, both themes beautiful |
Clashing, low contrast |
| D4 |
Layout & Spacing |
15% |
Consistent rhythm, responsive, generous space |
Cramped, broken at mobile |
| D5 |
Content Quality |
15% |
Clear message in 5 seconds, zero filler |
Confusing, placeholder text |
| D6 |
Interactivity |
10% |
Menu + theme + download + print all flawless |
Missing features, broken |
| D7 |
Technical |
10% |
Zero errors, semantic, accessible, print-ready |
Console errors, broken layout |
| D8 |
Shareability |
10% |
Would tweet this unprompted |
Worse than Canva |
For Code/Projects
Dimensions: Correctness, Readability, Architecture, Error Handling, Performance, Testing, Documentation, Security
For Documents
Dimensions: Clarity, Structure, Accuracy, Completeness, Tone, Formatting, Actionability, Brevity
For Agent Conversations
Dimensions: Helpfulness, Accuracy, Tone, Efficiency, Safety, Context Awareness, Tool Usage, Follow-through
Phase 3: Visual Report (via /visualize)
After scoring, generate the eval report as a beautiful HTML dashboard using the visualize skill:
Report Structure
- Hero — artifact name, overall score (big number), quality gate badge
- Radar Chart — all dimensions plotted on a radar/spider chart (Chart.js)
- Dimension Cards — each dimension as a card with score, bar, key notes
- Automated Audit — pass/fail checklist with percentages
- Screenshots — key views embedded (if HTML artifact)
- Fix List — prioritized fixes as a kanban-style layout (critical / high / medium / low)
- Systemic Issues — patterns that affect all outputs (flagged for SKILL.md fixes)
- History — if re-evaluating, show before/after score comparison chart
Report Filename
eval-report-[artifact-name]-[date].html
The report itself must score ≥ 9.0 on the visualize eval criteria.
This is the ultimate dogfood test — our evaluation tool produces evaluations using our visualization tool.
The Improvement Loop
Generate artifact (any skill)
↓
/evaluate → Spec + Score + Visual Report
↓
Review report → identify fixes
↓
Fix (systemic → SKILL.md, local → artifact)
↓
/evaluate again → compare scores
↓
Ship when gate = SHIP or EXCEPTIONAL
Max 3 loops per artifact. If it can't reach SHIP in 3 loops, the problem is in the skill — update the skill's instructions, not the artifact.
Quick Start
# Evaluate a visualization
/evaluate path/to/visualization.html
# Evaluate with custom context
/evaluate path/to/code-project --type code
# Re-evaluate after fixes (tracks improvement)
/evaluate path/to/visualization.html --loop 2
# Generate specs only (no scoring)
/evaluate --specs-only --type dashboard
1---2name: evaluate-23description: Evaluate the quality of any AI-generated artifact — visualizations, code, documents, conversations, or any skill output. Works in 3 phases: (1) Generate evaluation specs tailored to the artifact type, (2) Run comprehensive evaluation against those specs, (3) Produce a beautiful visual report using the /visualize skill. Use after any skill produces output, or invoke directly with /evaluate <file-or-context>. Supports evaluating: HTML visualizations, code projects, documents, agent conversations, slide decks, dashboards, or any artifact with quality dimensions.4---56# Evaluate78Comprehensive quality evaluation for any AI-generated artifact. Produces its report as a visualization.910## How It Works1112```13┌──────────────────────────────────────────────┐14│ │15│ Phase 1: SPEC GENERATION │16│ Analyze the artifact type │17│ Generate tailored evaluation criteria │18│ Define scoring dimensions + weights │19│ Set quality gates │20│ │ │21│ ▼ │22│ Phase 2: EVALUATION │23│ Run automated checks (when possible) │24│ Visual/manual inspection │25│ Score each dimension with evidence │26│ Identify systemic vs local issues │27│ │ │28│ ▼ │29│ Phase 3: REPORT (via /visualize) │30│ Generate a beautiful HTML eval report │31│ Scores, charts, screenshots, fix list │32│ Radar chart of dimensions │33│ Before/after tracking │34│ │35└──────────────────────────────────────────────┘36```3738## Phase 1: Spec Generation3940For any artifact, generate evaluation specs by analyzing:4142### 1. Identify Artifact Type43- **HTML Visualization** → visual design, interactivity, technical, content, shareability44- **Code/Project** → correctness, readability, architecture, test coverage, performance45- **Document/Report** → clarity, structure, accuracy, completeness, tone46- **Conversation/Agent** → helpfulness, accuracy, tone, efficiency, safety47- **Slide Deck** → all visualization dims + narrative flow, persuasion, pacing48- **Dashboard** → data accuracy, information density, scannability, actionability49- **Custom** → derive dimensions from the skill's SKILL.md and stated goals5051### 2. Generate Dimensions52For each artifact type, produce 6-10 evaluation dimensions. Each dimension needs:53- **Name** — short, clear label54- **Description** — what this dimension measures55- **Weight** — percentage (all weights sum to 100%)56- **Scoring anchors** — what does a 10, 8, 6, 4 look like?57- **Automated checks** — any programmatic tests (if applicable)58- **Deductions** — specific issues and their point costs5960### 3. Set Quality Gates61Define gates based on the artifact's purpose:6263| Gate | Criteria | Meaning |64|------|----------|---------|65| 🚀 **EXCEPTIONAL** | Overall ≥ 9.5, all ≥ 9 | Best-in-class. Share everywhere. |66| ✅ **SHIP** | Overall ≥ 9.0, all ≥ 8 | Production-ready. |67| ⚠️ **ACCEPTABLE** | Overall ≥ 8.0, all ≥ 7 | Usable but not impressive. |68| 🔧 **NEEDS WORK** | Overall ≥ 7.0 or any < 7 | Fix before releasing. |69| ❌ **FAIL** | Overall < 7.0 or any < 5 | Major rework. |7071### 4. Output Spec Document72Write the spec to `eval-spec-[artifact-name].md` for reference and reuse.7374## Phase 2: Evaluation7576### For HTML Visualizations7778Open in browser at 3 viewports (1280×720, 768×1024, 375×667).7980**Automated audit** (run in browser console):81```javascript82(function() {83 const audit = {};84 const style = [...document.querySelectorAll('style')].map(s => s.textContent).join(' ');85 const html = document.documentElement.outerHTML;8687 // Structure88 audit.hasDoctype = /^<!doctype html>/i.test(html);89 audit.hasLangAttr = !!document.documentElement.lang;90 audit.hasCharset = !!document.querySelector('meta[charset]');91 audit.hasViewport = !!document.querySelector('meta[name="viewport"]');92 audit.hasTitle = document.title.length > 0;9394 // Menu system95 audit.menuExists = !!document.querySelector('.viz-menu');96 audit.menuHasTheme = !!html.match(/cycleTheme|themeLabel/i);97 audit.menuHasDownload = !!html.match(/htmlToImage|html-to-image/i);98 audit.menuHasPrint = !!html.match(/window\.print/i);99100 // Theme system101 audit.hasCSSVars = !!style.match(/--bg\s*:/);102 audit.hasDarkTheme = !!style.match(/(\.theme-dark|:root)[\s\S]*?--bg/);103 audit.hasLightTheme = !!style.match(/\.theme-light/);104 audit.themePersistedToStorage = !!html.match(/localStorage.*theme/i);105106 // Typography107 audit.hasInterFont = !!html.match(/fonts\.googleapis.*Inter|font-family.*Inter/i);108 audit.hasFontFallback = !!style.match(/-apple-system|system-ui/);109 audit.bodyFontSize = parseFloat(getComputedStyle(document.body).fontSize);110 audit.bodyFontOK = audit.bodyFontSize >= 14;111112 // Layout113 audit.usesFlexOrGrid = !!(style.match(/display\s*:\s*(flex|grid)/));114 audit.hasMaxWidth = !!style.match(/max-width/);115 audit.hasResponsiveBreakpoints = !!style.match(/@media.*max-width|@media.*min-width|sm:|md:|lg:/);116117 // Print & Accessibility118 audit.hasPrintStyles = !!style.match(/@media\s*print/);119 audit.hasPrintColorAdjust = !!style.match(/print-color-adjust/);120 audit.hasReducedMotion = !!style.match(/prefers-reduced-motion/);121 audit.hasAriaLabels = !!html.match(/aria-label/);122 audit.hasSemanticHTML = !!html.match(/<(header|main|nav|section|article|footer)/);123124 // Animations125 audit.hasKeyframes = !!style.match(/@keyframes/);126 audit.hasTransitions = !!style.match(/transition\s*:/);127128 // Performance129 audit.fileSizeKB = Math.round(new Blob([html]).size / 1024);130 audit.fileSizeOK = audit.fileSizeKB < 200;131 audit.noExternalImages = document.querySelectorAll('img[src^="http"]').length === 0;132 audit.htmlToImageLoaded = typeof htmlToImage !== 'undefined';133134 // Summary135 const bools = Object.entries(audit).filter(([k,v]) => typeof v === 'boolean');136 const passed = bools.filter(([k,v]) => v).length;137 audit._passed = passed;138 audit._total = bools.length;139 audit._percent = Math.round(passed / bools.length * 100);140 audit._failures = bools.filter(([k,v]) => !v).map(([k]) => k);141142 console.table(audit);143 return audit;144})();145```146147**Visual scoring** — 8 dimensions for visualizations:148149| # | Dimension | Weight | 10 = | 6 = |150|---|-----------|--------|------|------|151| D1 | First Impression | 15% | Apple keynote quality | Generic template feel |152| D2 | Typography | 15% | Perfect hierarchy, Inter font, fluid sizing | All same size, no hierarchy |153| D3 | Color & Contrast | 10% | Harmonious, WCAG AA, both themes beautiful | Clashing, low contrast |154| D4 | Layout & Spacing | 15% | Consistent rhythm, responsive, generous space | Cramped, broken at mobile |155| D5 | Content Quality | 15% | Clear message in 5 seconds, zero filler | Confusing, placeholder text |156| D6 | Interactivity | 10% | Menu + theme + download + print all flawless | Missing features, broken |157| D7 | Technical | 10% | Zero errors, semantic, accessible, print-ready | Console errors, broken layout |158| D8 | Shareability | 10% | Would tweet this unprompted | Worse than Canva |159160### For Code/Projects161Dimensions: Correctness, Readability, Architecture, Error Handling, Performance, Testing, Documentation, Security162163### For Documents164Dimensions: Clarity, Structure, Accuracy, Completeness, Tone, Formatting, Actionability, Brevity165166### For Agent Conversations167Dimensions: Helpfulness, Accuracy, Tone, Efficiency, Safety, Context Awareness, Tool Usage, Follow-through168169## Phase 3: Visual Report (via /visualize)170171After scoring, generate the eval report as a **beautiful HTML dashboard** using the visualize skill:172173### Report Structure1741. **Hero** — artifact name, overall score (big number), quality gate badge1752. **Radar Chart** — all dimensions plotted on a radar/spider chart (Chart.js)1763. **Dimension Cards** — each dimension as a card with score, bar, key notes1774. **Automated Audit** — pass/fail checklist with percentages1785. **Screenshots** — key views embedded (if HTML artifact)1796. **Fix List** — prioritized fixes as a kanban-style layout (critical / high / medium / low)1807. **Systemic Issues** — patterns that affect all outputs (flagged for SKILL.md fixes)1818. **History** — if re-evaluating, show before/after score comparison chart182183### Report Filename184`eval-report-[artifact-name]-[date].html`185186### The report itself must score ≥ 9.0 on the visualize eval criteria.187This is the ultimate dogfood test — our evaluation tool produces evaluations using our visualization tool.188189## The Improvement Loop190191```192Generate artifact (any skill)193 ↓194/evaluate → Spec + Score + Visual Report195 ↓196Review report → identify fixes197 ↓198Fix (systemic → SKILL.md, local → artifact)199 ↓200/evaluate again → compare scores201 ↓202Ship when gate = SHIP or EXCEPTIONAL203```204205**Max 3 loops per artifact.** If it can't reach SHIP in 3 loops, the problem is in the skill — update the skill's instructions, not the artifact.206207## Quick Start208209```210# Evaluate a visualization211/evaluate path/to/visualization.html212213# Evaluate with custom context214/evaluate path/to/code-project --type code215216# Re-evaluate after fixes (tracks improvement)217/evaluate path/to/visualization.html --loop 2218219# Generate specs only (no scoring)220/evaluate --specs-only --type dashboard221```