optimize
Measure and improve your AgentCore agent's quality through evaluation, monitoring, and observability.
When to use
- You want to know if your agent is giving good answers
- You want to set up continuous quality monitoring in production
- You want to add a quality gate to your CI/CD pipeline
- You want to understand agent behavior through logs, metrics, and traces
- You want to set up CloudWatch dashboards or X-Ray tracing
Do NOT use for:
- Debugging a specific broken agent (wrong answers, errors) → use
agents-debug
- Production security hardening (IAM, auth) → use
agents-harden
Input
$ARGUMENTS can be:
- An eval goal: "add a quality gate", "set up monitoring"
- An observability goal: "set up CloudWatch dashboard", "understand my traces"
- A specific evaluator: "llm-as-a-judge", "code-based"
- Empty — the skill will guide based on project context
Process
Step 0: Verify CLI version
Run agentcore --version. This skill requires v0.9.0 or later.
Step 1: Read project context
Read agentcore/agentcore.json to understand existing evaluators, online eval configs, and agent setup.
If agentcore/agentcore.json is not found:
"This skill requires an AgentCore project. Use agents-get-started to create one."
Step 2: Determine the workflow
| Developer intent |
Action |
| Measure quality, add evaluator, run eval, CI/CD gate, online monitoring |
Load references/evals.md and follow its workflow |
| Set up observability, CloudWatch, X-Ray, logs, metrics, dashboards |
Load references/observability.md and follow its workflow |
| Understand or reduce AgentCore costs |
Load references/cost.md |
| Both — "I want to understand and improve my agent" |
Start with observability setup, then add evals |
Step 3: Follow the loaded reference
The reference file contains the full procedure. Follow it step by step.
Cross-references
- After setting up evals, suggest
agents-harden for production readiness
- If eval results reveal agent issues, suggest
agents-debug for root cause analysis
- If the developer needs to add capabilities first, suggest
agents-build
Output
Depends on the workflow — see the loaded reference for specific outputs.
Quality criteria
- Evaluator configuration uses only valid CLI flags
- Online eval sampling rate is appropriate (not 100% in production without discussion)
- CI/CD quality gate has a clear pass/fail threshold
- Observability setup includes both tracing and logging
- The developer understands the eval data delay: ~10 seconds put-to-get, end-to-end — one ingestion step covers both trace reads and eval queries; there is no separate indexing wait
1---
2name: agents-optimize
3description: >
4 Use when measuring or improving agent quality and performance — set up
5 evaluators, online monitoring, CI/CD quality gates, observability, or
6 cost optimization. Triggers on: "evaluate my agent", "add evaluator",
7 "measure quality", "quality gate", "run evals", "agent too slow",
8 "why is it slow", "reduce latency", "set up observability", "CloudWatch
9 dashboard", "how much does my agent cost", "cost optimization", "logs
10 not showing up", "logs missing", "spans not found", "eval failing",
11 "eval error", "dev traces", "local traces", "agentcore dev traces",
12 "traces to CloudWatch".
13 Not for debugging errors or crashes — use agents-debug. Slow but
14 correct routes here; broken routes to debug.
15allowed-tools: Read Grep Glob Bash
16metadata:
17 type: skill
18 version: "1.0.0"
19 author: aws-agentcore
20 requires-cli: ">=0.9.0"
21---
22
23# optimize
24
25Measure and improve your AgentCore agent's quality through evaluation, monitoring, and observability.
26
27## When to use
28
29- You want to know if your agent is giving good answers
30- You want to set up continuous quality monitoring in production
31- You want to add a quality gate to your CI/CD pipeline
32- You want to understand agent behavior through logs, metrics, and traces
33- You want to set up CloudWatch dashboards or X-Ray tracing
34
35Do NOT use for:
36
37- Debugging a specific broken agent (wrong answers, errors) → use `agents-debug`
38- Production security hardening (IAM, auth) → use `agents-harden`
39
40## Input
41
42`$ARGUMENTS` can be:
43
44- An eval goal: "add a quality gate", "set up monitoring"
45- An observability goal: "set up CloudWatch dashboard", "understand my traces"
46- A specific evaluator: "llm-as-a-judge", "code-based"
47- Empty — the skill will guide based on project context
48
49## Process
50
51### Step 0: Verify CLI version
52
53Run `agentcore --version`. This skill requires v0.9.0 or later.
54
55### Step 1: Read project context
56
57Read `agentcore/agentcore.json` to understand existing evaluators, online eval configs, and agent setup.
58
59If `agentcore/agentcore.json` is not found:
60> "This skill requires an AgentCore project. Use `agents-get-started` to create one."
61
62### Step 2: Determine the workflow
63
64| Developer intent | Action |
65|---|---|
66| Measure quality, add evaluator, run eval, CI/CD gate, online monitoring | Load [`references/evals.md`](references/evals.md) and follow its workflow |
67| Set up observability, CloudWatch, X-Ray, logs, metrics, dashboards | Load [`references/observability.md`](references/observability.md) and follow its workflow |
68| Understand or reduce AgentCore costs | Load [`references/cost.md`](references/cost.md) |
69| Both — "I want to understand and improve my agent" | Start with observability setup, then add evals |
70
71### Step 3: Follow the loaded reference
72
73The reference file contains the full procedure. Follow it step by step.
74
75### Cross-references
76
77- After setting up evals, suggest `agents-harden` for production readiness
78- If eval results reveal agent issues, suggest `agents-debug` for root cause analysis
79- If the developer needs to add capabilities first, suggest `agents-build`
80
81## Output
82
83Depends on the workflow — see the loaded reference for specific outputs.
84
85## Quality criteria
86
87- Evaluator configuration uses only valid CLI flags
88- Online eval sampling rate is appropriate (not 100% in production without discussion)
89- CI/CD quality gate has a clear pass/fail threshold
90- Observability setup includes both tracing and logging
91- The developer understands the eval data delay: **~10 seconds put-to-get, end-to-end** — one ingestion step covers both trace reads and eval queries; there is no separate indexing wait
aws