ag2ai AG2 — capability map
AG2 (ag2) is an async, protocol-driven agent framework. The full reference docs live under website/docs/user-guide/. This skill is the index of sibling skills that cover the common build paths.
When to use
Read this file first when a request mentions "AG2", "ag2", or building agents in this repo and you don't yet know which feature is needed. Use the table below to pick the right specialised skill, then load that skill's SKILL.md for the recipe.
Before you start
Anything you build with AG2 needs three things in place. Get these right once and the rest of the skills run cleanly:
- Install the right provider extra —
pip install "ag2[openai]",ag2[anthropic],ag2[gemini], etc. The*Configclass will raiseImportError: ... requires optional dependencieswithout it. Run the install before delivering code. If you cannot run commands, state the exactpip installcommand. This is a required step. - Set the matching API key —
OPENAI_API_KEY,ANTHROPIC_API_KEY,GEMINI_API_KEY(orGOOGLE_API_KEY). Loading from a project-root.envviafrom dotenv import load_dotenv; load_dotenv()is the common pattern. - Sanity-check the install —
python -c "import sys, ag2; from importlib.metadata import version; print(sys.executable, version('ag2'))". If you have multiple Python environments, this confirms whichag2your script will actually import.
Full per-provider table (install + env var + config class) lives in ag2-quickstart → "Prerequisites".
Pick the right skill
| User intent | Skill | What it covers |
|---|---|---|
Build an Agent from scratch, pick a model |
ag2-quickstart |
Agent, ModelConfig, ask() / reply.ask() chaining, providers, env vars |
| Give the Agent a custom Python tool | ag2-add-custom-tool |
@tool, sync/async, ToolResult, Context, Inject, Variable, Depends |
| Use shipped tools (web search, code exec, MCP, etc.) | ag2-use-builtin-tools |
WebSearchTool, WebFetchTool, CodeExecutionTool, MCPServerTool, ImageGenerationTool, MemoryTool, FilesystemToolkit, DuckDuckSearchTool, ExaToolkit, TavilySearchTool |
| Run shell commands from an agent | ag2-shell-tool |
SandboxShellTool + LocalEnvironment (any provider), provider-side ShellTool (OpenAI Responses), sandboxing (allowed/blocked/ignore/readonly) |
| Get typed Pydantic / dataclass output | ag2-structured-output |
response_schema=, ResponseSchema, @response_schema, PromptedSchema, reply.content(), retries |
| Multi-agent: parallel subtasks or named delegates | ag2-subagent-delegation |
tasks=TaskConfig(), run_subtasks(parallel=True), Agent.as_tool(), persistent_stream |
| Pause for human input or gate a tool with approval | ag2-hitl |
context.input(), hitl_hook, approval_required() middleware |
| Logging, retry, history-trim, custom interception | ag2-middleware |
BaseMiddleware, LoggingMiddleware, RetryMiddleware, HistoryLimiter, TokenLimiter, tool middleware |
| Test agents and tools | ag2-testing |
TestConfig, mocking LLM responses, simulating ToolCallEvent |
| Evaluate / benchmark an agent offline, CI gate, scorers | ag2-evaluation |
Suite, run_agent, final_answer_matches / tool_called / agent_judge, @scorer, RunResult, TestConfig, evaluate_traces, diff().regressions |
| Compare models / prompts — leaderboard or head-to-head | ag2-eval-comparison |
run_variants (Variants({name: Agent}, axis=...)), run_pairwise + pairwise_judge (win-rate, Wilson CI, flips, κ), human_pairwise |
| Persistent memory across runs, history compaction, assembly | ag2-knowledge-and-memory |
KnowledgeStore, KnowledgeConfig, WorkingMemoryAggregate, AssemblyPolicy, SlidingWindowPolicy, TokenBudgetPolicy, TailWindowCompact, SummarizeCompact |
| Observability, alerts, halts | ag2-observers-and-alerts |
BaseObserver, TokenMonitor, LoopDetector, EventWatch, CadenceWatch, AlertPolicy, HaltEvent |
| Send images / audio / video / PDFs in | ag2-multimodal-input |
ImageInput, AudioInput, VideoInput, DocumentInput, FilesAPI |
| Web frontend via the AG-UI protocol | ag2-ag-ui |
AGUIStream, FastAPI mount, CopilotKit |
| Serve an agent as an MCP server (server side) | ag2-mcp |
MCPServer, stdio / HTTP, prompts, resources, SessionConfig, AskContext |
| Expose an agent over the A2A protocol | ag2-a2a |
A2AServer, build_jsonrpc / build_rest / build_grpc, AgentCard, A2AConfig (consume remote) |
| Realtime voice / live audio (talking agent, STT, TTS) | ag2-live |
LiveAgent, GeminiRealTimeConfig / OpenAIRealTimeConfig, SoundDeviceRecorder / Player, OpenAITranscriber, TTSObserver |
| OpenTelemetry traces / metrics | ag2-telemetry |
TelemetryMiddleware, GenAI semconv attributes, content capture |
Multi-agent networks
Whenever two or more agents need to interact, load ag2-network-quickstart first — the network is the standard multi-agent pattern in AG2. It covers the Hub setup and the two 2-party channel adapters (consulting for strict 1Q1R and conversation for free-form). After the quickstart, route to the right deep-dive:
| User intent | Skill | What it covers |
|---|---|---|
| N-party round-robin / fixed turn order | ag2-network-discussion |
discussion adapter, ORDERING_ROUND_ROBIN knob, can_send probe pattern, view-window sizing |
Declarative orchestration / TransitionGraph / GroupChat migration |
ag2-network-workflow |
workflow adapter, TransitionGraph.sequence / .round_robin, Handoff, ToolCalled, ContextEquals, context_vars, 8 cookbook patterns, classic-GroupChat migration |
| Rate limits, access policy, expectations, audit, capability tracking | ag2-network-governance |
Rule (AccessBlock / LimitsBlock / RateBlock / InboxBlock), Expectations, audit log + AUDIT_KIND_*, task observation, Resume.observed |
| Custom envelope handlers, view policies, peer discovery, the LLM-facing network tools | ag2-network-tools-and-views |
adapter-owned say + the five NetworkPlugin tools (delegate / peers / channels / tasks / context), agent_client.on_envelope, ViewPolicy (FullTranscript / WindowedSummary / NamedWindowedSummary), skill_md peer discovery, full Envelope / EV_* reference |
Not a network task: if the user has one agent recursively spawning its own sub-tasks (via run_subtask / run_subtasks(parallel=True)) or calling another agent as a lightweight tool, use ag2-subagent-delegation — no hub, no registry, no channels. The network is for distinct, registered agents collaborating through a shared hub.