Holmesgpt Eval

Run HolmesGPT evals (tests/llm/fixtures/test_ask_holmes) against the current agent itself (agent-native). The agent must answer the prompts itself (no external wrapper) using its own tools, run before_test/after_test around each case, compare the answer to expected_output using the scoring rules here, and generate JSON + Markdown reports. This skill focuses exclusively on the agent-native flow. Supports bulk run (all tests) and individual run (specific test case). Supports filtering tests by tags.

Azure Updated 282 repo stars

File contents

azure/aks-skills commit 634f8fe6a7

Frequently asked questions

npx get-skill add azure/holmesgpt-eval