Report

ACED Report

Generate a project-wide health dashboard across all eval suites.

Discover eval suites

Scan the project spec (.agents/specs/) for behavioral-leaf nodes carrying an eval.md (subject + run policy) alongside their <node>.feature. For each, read:

• eval.md for target path and threshold
• The most recent file in results/ (sort by filename descending)
• The second-most-recent file in results/ for trend (if it exists)

If no matching directories are found, report that no eval suites are initialized and suggest sdd:start-mission (the conductor resolves the ACED roles).

Compute per-suite metrics

For each suite:

• Pass rate (latest run)

• Pass rate delta vs. previous run (trend)

• Worst failing case: its name, and its total against its own maximum (3/5 vs 4)

• Mean %max — the mean of each rubric scenario's total ÷ its own maximum, as a percentage

Never mean the raw totals. Each scenario's rubric declares its own per-dimension maxima, so totals from different scenarios sit on different scales and averaging them measures nothing. Normalize first: total ÷ max per scenario, then mean those fractions. Always label the column %max so it is never read as a raw score. A suite with no rubric scenarios (all boolean or trigger) has no %max — render —.

Classify health

Report

ACED Project Report
═══════════════════════════════════════════════

  Suite                  Status       Pass   Mean %max   Trend
  ───────────────────────────────────────────────────────────
  commit-discipline      healthy      95%    92%         ↑ +5%
  create-skill           degraded     76%    74%         → 0%
  aced-case-judge        critical     58%    61%         ↓ -12%
  subagents/researcher   no-data       —      —          —

NEEDS ATTENTION:
  ✗ aced-case-judge (critical, trending down) — run aced-improve
      worst: red tests block the commit  1/5 vs 4  (correctness 0/3, completeness 1/2)
  ⚠ create-skill (degraded) — run run for details

Suites with no results: subagents/researcher — run run to initialize

Optional: full detail mode

If the user asks for details on a specific suite, print all failing cases from the latest result file with their per-dimension scores, their total against its maximum, the threshold, and the case's what_failed.

Suggest next actions

• For critical or trending-down: suggest improve
• For degraded: suggest run for details, then improve
• For no-data: suggest run
• For all healthy with no trend data: suggest add-scenario to expand coverage