Purpose
Cresta (a customer experience AI company) built Conductor, a natural-language agent builder that
helps teams create production-ready CX agents. Users describe what they want, and Conductor guides
the work through a blueprint, implementation, evaluation, and optimization. Early use across Cresta
and its partners reportedly cut initial deployment time roughly in half. This skill turns that
approach into a repeatable plan for any domain.
When to use
- You have domain data and expertise (e.g. historical conversations) and want to package it as an
agent builder.
- You are deciding what to build yourself versus take off the shelf on the Claude Agent SDK.
- You need an evaluation loop that tells you what improved or regressed when a new Claude model
ships or your framework changes.
Instructions
Split the stack. Use the Claude Agent SDK as the general-purpose execution harness: gathering
context, calling tools, writing and running code. Put your domain workflows, tools, and domain
context on top. Invest only in what only you can provide (for Cresta: conversation data and CX
expertise); use off-the-shelf components for the rest.
Check enterprise requirements before committing. Cresta evaluated the SDK for data privacy,
tenant architecture, organization-level key distribution, controls, and observability. Record
your own answers for each.
Model the builder flow. Guide the user from a natural-language description through four
stages: blueprint → implementation → evaluation → optimization.
Decide what must be deterministic. Use historical data (for Cresta, past conversations) to
find where a human should stay in the loop and where the model can adapt. Harden the workflows
that can never fail — especially in regulated industries — with explicit rules. Keep the rest
flexible. Test and revise those rules as business requirements change.
Build a fixed evaluation suite of build tasks. Cresta's tasks: create an agent, write a test
case, change an existing agent, perform root cause analysis. Score every run on four dimensions:
- Outcome — was a usable artifact produced?
- Execution path — were appropriate tools and context used?
- Quality — how does the result compare with a curated reference?
- Resource use — time and model usage.
Use templates/build-task-scorecard.md.
Rerun on every change. When a new Claude model launches or your framework changes, rerun the
same tasks with the same scoring and compare against the previous run to see what improved and
what regressed.
Dogfood before shipping. Use the builder internally first (Cresta started with its own
deployment teams, using Claude Sonnet and Claude Opus). Treat it as a product only when its
output matches what an expert engineer would build.
Start the knowledge flywheel. Let builders capture proven patterns, business rules, and
evaluation approaches as reusable memory artifacts, and share them as skills across the team.
Budget for iteration. Cresta puts the initial build at about 20% of the effort; testing,
optimization, and continuous improvement are the other 80%.
Examples
Contact-center agent for a regulated industry
- Deterministic: identity verification and required disclosures are explicit rules with tests.
- Flexible: small talk, rephrasing, and order of non-critical questions.
- Historical conversations show that refund disputes above a threshold need a human in the loop.
Model upgrade check
- Rerun the four build tasks on the new model with the same scorecard.
- Result: outcome and quality improve on "change an existing agent", resource use rises on
"root cause analysis" — investigate before switching the default.
Notes
- The post does not publish Cresta's rule language, task fixtures, or scoring thresholds; define
your own. See the original article for context.
1---2name: domain-agent-builder3description: Design a natural-language agent builder on top of the Claude Agent SDK, following the approach Cresta used for Conductor, its CX agent builder — the SDK as the general-purpose execution harness, your domain workflows, tools, and context layered on top, a blueprint → implementation → evaluation → optimization flow, explicit decisions about which workflows must be deterministic, a fixed build-task evaluation suite rerun on every model or framework change, and reusable memory artifacts shared as skills. Use when a team with deep domain expertise wants to turn it into a product that builds production-ready agents.4metadata:5 source: https://skillmd.com/skills/uygnoey/domain-agent-builder6---78## Purpose9Cresta (a customer experience AI company) built Conductor, a natural-language agent builder that10helps teams create production-ready CX agents. Users describe what they want, and Conductor guides11the work through a blueprint, implementation, evaluation, and optimization. Early use across Cresta12and its partners reportedly cut initial deployment time roughly in half. This skill turns that13approach into a repeatable plan for any domain.1415## When to use16- You have domain data and expertise (e.g. historical conversations) and want to package it as an17 agent builder.18- You are deciding what to build yourself versus take off the shelf on the Claude Agent SDK.19- You need an evaluation loop that tells you what improved or regressed when a new Claude model20 ships or your framework changes.2122## Instructions23241. **Split the stack.** Use the Claude Agent SDK as the general-purpose execution harness: gathering25 context, calling tools, writing and running code. Put your domain workflows, tools, and domain26 context on top. Invest only in what only you can provide (for Cresta: conversation data and CX27 expertise); use off-the-shelf components for the rest.28292. **Check enterprise requirements before committing.** Cresta evaluated the SDK for data privacy,30 tenant architecture, organization-level key distribution, controls, and observability. Record31 your own answers for each.32333. **Model the builder flow.** Guide the user from a natural-language description through four34 stages: blueprint → implementation → evaluation → optimization.35364. **Decide what must be deterministic.** Use historical data (for Cresta, past conversations) to37 find where a human should stay in the loop and where the model can adapt. Harden the workflows38 that can never fail — especially in regulated industries — with explicit rules. Keep the rest39 flexible. Test and revise those rules as business requirements change.40415. **Build a fixed evaluation suite of build tasks.** Cresta's tasks: create an agent, write a test42 case, change an existing agent, perform root cause analysis. Score every run on four dimensions:43 - **Outcome** — was a usable artifact produced?44 - **Execution path** — were appropriate tools and context used?45 - **Quality** — how does the result compare with a curated reference?46 - **Resource use** — time and model usage.47 Use [templates/build-task-scorecard.md](./templates/build-task-scorecard.md).48496. **Rerun on every change.** When a new Claude model launches or your framework changes, rerun the50 same tasks with the same scoring and compare against the previous run to see what improved and51 what regressed.52537. **Dogfood before shipping.** Use the builder internally first (Cresta started with its own54 deployment teams, using Claude Sonnet and Claude Opus). Treat it as a product only when its55 output matches what an expert engineer would build.56578. **Start the knowledge flywheel.** Let builders capture proven patterns, business rules, and58 evaluation approaches as reusable memory artifacts, and share them as skills across the team.59609. **Budget for iteration.** Cresta puts the initial build at about 20% of the effort; testing,61 optimization, and continuous improvement are the other 80%.6263## Examples6465**Contact-center agent for a regulated industry**66- Deterministic: identity verification and required disclosures are explicit rules with tests.67- Flexible: small talk, rephrasing, and order of non-critical questions.68- Historical conversations show that refund disputes above a threshold need a human in the loop.6970**Model upgrade check**71- Rerun the four build tasks on the new model with the same scorecard.72- Result: outcome and quality improve on "change an existing agent", resource use rises on73 "root cause analysis" — investigate before switching the default.7475## Notes76- The post does not publish Cresta's rule language, task fixtures, or scoring thresholds; define77 your own. See the original article for context.