Nat Evaluation

Use when designing, configuring, running, or troubleshooting NeMo Agent Toolkit evaluations, datasets, evaluator selection, ATIF surfaces, quality gates, custom evaluators, and `nat eval`.

NVIDIA Updated 2.2k repo stars

File contents

NeMo Agent Toolkit Evaluation

Use this skill for measuring agent quality and behavior.

Workflow

  1. Decide the evaluation surface and output format.
  2. Decompose quality goals into separate evaluators.
  3. Choose built-in evaluators before writing custom evaluators.
  4. Keep datasets small and explicit for local validation.
  5. Run nat eval and inspect generated artifacts.

References

  • references/operating-mode.md
  • references/methodology.md
  • references/agent-eval-framework.md
  • references/evaluation-surfaces.md
  • references/evaluation-contract.md
  • references/evaluators/
  • references/code-patterns.md

NVIDIA/NeMo-Agent-Toolkit/tree/main/skills/nat-evaluation commit 9ed47e4e74

Frequently asked questions

npx skillmds@latest add nvidia/nat-evaluation