# Post Eval

> Runs a post-batch analysis pipeline after an eval completes: verifies results, executes analysis scripts, refreshes dashboards, and writes a summary report.

- Skill: `samyakjhaveri/post-eval` (Agent Skill)
- Install (CLI): `npx skillmds add samyakjhaveri/post-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/samyakjhaveri/post-eval/raw
- Safety review: CAUTION (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra, CI/CD
- Tags: Analysis, Dashboard, Eval, Post Processing, Summary
- Author: SamyakJhaveri (https://skillmd.com/u/samyakjhaveri)
- Updated: 2026-08-22
- Page: https://skillmd.com/skills/samyakjhaveri/post-eval

---


# Post-Eval Pipeline

After an eval batch completes, run the full post-processing chain: verify results,
analyze, classify, refresh dashboard, and generate a summary. Each stage validates
the previous stage's output before proceeding.

**Trigger:** When user types `/post-eval` or `/post-eval <model-name>`

## Arguments

- `$ARGUMENTS` — optional model name filter. If omitted, analyze all models.

## Critical Rules

1. **Ground truth is result files, not summaries.** Always verify claims against
   the actual result files on disk.
2. **Exclusion list is non-negotiable.** Apply known-failing exclusions consistently.
3. **Never assume "all clean" without spot-checking.** Check 5-10 results manually.
4. **Classify failures granularly.** Different failure types have different root causes
   and implications for the paper.
5. **Check experiment configuration explicitly.** Don't assume parameters are correct.

## Workflow

### Step 0: Pipeline Integrity Check

Before running analysis, verify the evaluation operated correctly:
1. Status consistency across result fields
2. Configuration verification (parameters match expectations)
3. Coverage check (expected tasks all present)
4. Exclusion list applied correctly

If any check fails, report before proceeding.

### Step 1: Verify Results Exist

Count result files per model/configuration. If no results, stop.

### Step 2: Run Analysis Scripts

Execute the project's analysis pipeline. Verify outputs were created.

### Step 3: Classification (if applicable)

Run any classification or categorization scripts. Verify output.

### Step 4: Refresh Dashboard (if applicable)

Update any visualization or dashboard files with new results.

### Step 5: Generate Summary

Write a structured summary to the project's docs directory:
- Overall pass rates by model
- Configuration breakdown
- Notable findings and anomalies
- Gaps in coverage

### Step 6: Report & Suggest Next Steps

```
=== POST-EVAL PIPELINE COMPLETE ===

Step 0: Integrity check  - PASS / FAIL
Step 1: Results verified  - <N> models, <N> total files
Step 2: Analysis          - PASS / FAIL
Step 3: Classification    - PASS / FAIL / SKIP
Step 4: Dashboard         - PASS / FAIL / NO_CHANGES
Step 5: Summary written   - <path>

Suggested next steps:
  [ ] Review summary for anomalies
  [ ] Run /validate before committing
  [ ] Update paper sections if relevant
```

## Error Handling

If any step fails:
1. Report the specific error
2. Mark that step as FAIL
3. Continue to next step if possible (steps are mostly independent after Step 2)
4. Do NOT retry automatically — let the user decide