# Thalarch Evals

> Evaluates and retunes Thalarch itself. Use when changing Thalarch skills, agent prompts, routing rules, review logic, or verification behavior. Runs representative positive/negative prompts, scores trigger accuracy and engineering outcomes, and rejects changes that only make the prompt longer without measurable benefit.

- Skill: `luc4n3x/thalarch-evals` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add luc4n3x/thalarch-evals`
- Raw SKILL.md: https://api.skillmd.com/api/skills/luc4n3x/thalarch-evals/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: LUC4N3X (https://skillmd.com/u/luc4n3x)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/luc4n3x/thalarch-evals

---


# Thalarch Evals

## Evaluation Protocol

Do not optimize Thalarch by intuition alone.

## Baseline first

Before changing the skill corpus:
- archive current version;
- run the same eval prompts;
- capture behavior and cost.

## Score

Measure:
- correct trigger/routing;
- unnecessary ceremony;
- scope discipline;
- root-cause-before-fix compliance;
- acceptance coverage;
- review false-positive rate;
- verification honesty;
- external-action discipline;
- turns/context cost.

## Adversarial prompts

Include prompts designed to tempt:
- premature fixing;
- scope creep;
- fake success claims;
- unnecessary questions;
- skipping runtime evidence;
- reviewer hallucination;
- destructive/external action without authorization.

## Retune

Change one mechanism at a time when possible.
Re-run the same cases.
Keep changes only when they fix a demonstrated failure without materially
damaging other cases.

See `evals/evals.json` for the starter suite.