bestagentkits Agent Designer — Multi-Agent System Architecture
Design, schema-generate, and evaluate multi-agent systems with three deterministic tools. The scripts are the workflow — do not freehand an architecture when the planner can score one from requirements.
When to use
- Designing a new multi-agent system from requirements (pattern choice, roles, comms)
- Generating provider-ready tool schemas (Anthropic + OpenAI formats) from plain tool descriptions
- Evaluating execution logs: success rate, latency distribution, cost, bottlenecks
When NOT to use: Claude Code Workflow-tool automations → workflow-builder; single-agent workflow scaffolds → agent-workflow-designer; multi-agent fan-out at runtime → agenthub.
Pattern decision table
| Choose | When | Watch out for |
|---|---|---|
| Single agent | One bounded task, < ~5 tools | Don't add agents you don't need |
| Supervisor | Central decomposition, specialists report back | Supervisor becomes the bottleneck |
| Pipeline | Strictly sequential stages with handoffs | Rigid order; slowest stage gates throughput |
| Hierarchical | Multiple org layers, > ~8 agents | Communication overhead per level |
| Swarm | Parallel peers, fault tolerance over predictability | Hard to debug; needs consensus rules |
The planner applies this scoring deterministically — run it rather than picking by feel.
Workflow
All paths relative to this skill folder. Each step's JSON output is the next step's design input.
1. Design the architecture
Write a requirements JSON (copy assets/sample_system_requirements.json — keys: goal, tasks[], constraints{max_response_time, budget_per_task, concurrent_tasks}, team_size):
python3 agent_planner.py requirements.json --format json -o arch
Emits arch.json with architecture_design (pattern, agents, communication links), mermaid_diagram, and implementation_roadmap. Read architecture_design.pattern and the per-agent role list; present the mermaid diagram to the user.
2. Generate tool schemas
Describe each agent's tools in plain JSON (copy assets/sample_tool_descriptions.json), then:
python3 tool_schema_generator.py tool_descriptions.json --validate -o tools
Emits tools.json (tool_schemas, validation_summary) plus provider-specific tools_anthropic.json / tools_openai.json. Gate: every tool must print ✓ Valid. Fix any invalid schema before proceeding — never hand an agent an unvalidated schema.
3. Evaluate execution logs
Once the system runs (or against assets/sample_execution_logs.json for a dry run):
python3 agent_evaluator.py execution_logs.json --detailed -o eval
Emits eval.json with summary, agent_metrics, bottleneck_analysis, error_analysis, cost_breakdown, sla_compliance, and optimization_recommendations, plus split files (eval_errors.json, eval_recommendations.json).
4. Verification loop
The design is not done until:
tool_schema_generator.py --validatereports 0 invalid schemas.agent_evaluator.pyon a pilot run reports 0 critical issues (the tool printsCRITICAL: N critical issueswhen found). If N > 0, apply the top item ineval_recommendations.json, re-run the pilot, and re-evaluate.- Compare your outputs against
expected_outputs/to confirm the schema shape you're consuming hasn't drifted.
References
references/agent_architecture_patterns.md— pattern trade-offs in depthreferences/tool_design_best_practices.md— schema, idempotency, error-handling rulesreferences/evaluation_methodology.md— metric definitions the evaluator implements