Oxy Evals

Use when writing, running, or acting on evals for an Oxy agent — `.test.yml` files that run an `.agentic.yml` agent and score its answers with an LLM judge — and when the agent's answers are wrong. Triggers: "add a test", "write a test case", "write evals", "test this agent", "evaluate my agent", "bootstrap an eval suite", "fill in expected answers", "run the tests"; and the repair side: "tests are failing", "this test is flaky", "the agent's answer is wrong", "the answer should be X but the agent says Y", "debug this eval", "make this test pass", or pasted `oxy test --output-json` output or a prompt/response pair to diagnose. Drafts expected answers from observed runs, then diagnoses failures and repairs the agent's semantic model, context and instructions — never the test.

oxy-hq Updated

File contents

oxy-hq/oxygen commit 534be51540

Frequently asked questions

npx get-skill add oxy-hq/oxy-evals