Research Multi Agent Systems

Research Multi-Agent Systems

Objective

Produce evidence-backed multi-agent research that separates genuine coordination effects from extra inference compute. Cover papers, official company systems, protocols, benchmarks, and executable code. Match the response language to the user.

Apply the Scope Test

Classify the target before searching:

1. Classical MAS: autonomous entities, distributed problem solving, negotiation, organizations, BDI, auctions, or agent communication languages.
2. MARL and games: multiple learning policies in a shared environment, including cooperative, competitive, or mixed settings.
3. LLM problem-solving teams: several model instances that propose, route, critique, debate, verify, or synthesize.
4. Generative societies: multiple LLM agents used to simulate people, institutions, markets, games, or emergent behavior.
5. Embodied teams: multiple robots or environment-grounded agents coordinating actions.
6. Interoperability and production: orchestration runtimes, agent-to-agent protocols, state, tracing, permissions, and deployment.

Do not silently collapse these areas into one literature list. Read taxonomy.md when the request requires definitions, architecture selection, or historical context.

Enforce Terminology

Use these distinctions:

• Call a system multi-agent when two or more policy-bearing entities have separate state, context, roles, observations, tools, or action opportunities and influence a joint outcome.
• Call independent parallel generations with one final aggregator an ensemble or independent-agent architecture. Note that it is the weakest interaction form and may not demonstrate collaboration.
• Call a fixed code path a workflow; call a model-directed path agentic orchestration. A system may combine both.
• Distinguish an agent system solving a task from a single agent acting in a multi-agent environment.
• Distinguish MCP for agent-to-tool/context connections from A2A for communication between independent agents. Treat IBM ACP as historical because it merged into A2A in 2025.
• Avoid “swarm” unless the work actually uses decentralized or population-level coordination, or the project uses that name.

Research Workflow

1. Fix the Question and Cutoff

Record:

• target domain and whether adjacent single-agent work is in scope;
• desired time range and a precise “checked through” date;
• required evidence: papers, company systems, code, benchmarks, or all;
• expected depth: canonical map, systematic search, implementation comparison, or experiment plan.

For “all work,” state that the result is a structured, reproducible map rather than a literal guarantee of exhaustiveness. Search at minimum cs.MA, cs.AI, cs.CL, and cs.LG, plus relevant venues such as AAMAS, NeurIPS, ICML, ICLR, ACL, EMNLP, AAAI, IJCAI, UIST/CHI, CoRL, ICRA, and IROS.

2. Retrieve Primary Evidence

Browse for current information. Prefer sources in this order:

1. peer-reviewed proceedings or official paper page;
2. arXiv/OpenReview when no archival version exists;
3. author or lab project page;
4. official author/company repository;
5. official company product documentation or engineering/research post.

Use surveys to expand coverage, not as the sole evidence for a method. For company claims, cite the company and label the claim as internal unless independently reproduced. For technical questions about OpenAI, use current official OpenAI documentation.

Search paper title, authors, year, venue, stable identifier, and code independently. Deduplicate arXiv, workshop, and conference versions as one work.

3. Build the Evidence Matrix

Track at least:

Read canonical-works.md for the seeded literature map and industry-systems.md for company work. Reverify anything described as current.

4. Select Canonical Work Transparently

Use three tiers:

• Foundational: introduced a durable formalism, interaction pattern, benchmark, or widely reused system.
• Canonical empirical: established a strong result, negative result, or common experimental baseline.
• Frontier/watchlist: recent, influential-looking, or preprint-only work needing more validation.

Rank using conceptual novelty, downstream influence, empirical rigor, code quality, and continued relevance. Do not rank by citation count or repository stars alone.

5. Inspect Code, Not Only the Badge

For important repositories, inspect the README and relevant source/configuration paths. Record:

• whether the repository is official and matches the cited version;
• active, maintenance, archived, or superseded status;
• exact methods and benchmarks implemented;
• model/provider coupling and whether local models are supported;
• environment requirements such as Docker, StarCraft II, browser sandboxes, paid APIs, or cloud accounts;
• test/evaluation harness, seeds, logs, configs, and license;
• gaps between the paper and released code.

Use codebases.md for starting points and migration warnings. Prefer MASLab for a unified comparison of many LLM-MAS methods, but verify its reproduction against each official implementation for results that matter.

6. Compare Against Honest Baselines

Always include:

1. strongest single agent with the same tools and context;
2. budget-matched independent sampling plus voting/selection;
3. single-agent self-refinement or verifier loop;
4. the proposed multi-agent topology;
5. an oracle or executable verifier when feasible.

Hold model family, tool access, context, and total inference budget constant where possible. Report both raw performance and quality per token, per call, per dollar, and per unit latency. Do not attribute a gain to coordination if added sampling or a stronger aggregator explains it.

Read evaluation-and-failures.md before recommending an experiment, benchmark, or production system.

7. Synthesize by Mechanism

Organize conclusions by the causal mechanism, not by paper chronology alone:

• parallel search and context isolation;
• specialization and access control;
• diversity and ensemble aggregation;
• critique, verification, and adversarial debate;
• dynamic routing, topology search, or learned orchestration;
• coordination through shared artifacts or environments;
• social emergence and strategic interaction;
• training-time cooperation or communication learning.

For each mechanism, state when it should work, when it should fail, the best evidence, the cost, and the simplest competing baseline.

Architecture Decision Rules

• Use independent parallel workers for breadth-first tasks with separable subproblems.
• Use manager-workers when one agent must own decomposition, budget, and final synthesis.
• Use agents-as-tools when a manager must retain final-answer ownership.
• Use handoffs when a specialist should take control of the next user-facing branch.
• Use a deterministic workflow when compliance, ordering, or side effects matter more than flexible planning.
• Use debate/critique only when independent views are likely and answers can be adjudicated; guard against conformity and correlated errors.
• Use shared artifacts with explicit ownership for coding and long-running work; avoid unrestricted shared-memory chat.
• Prefer a single agent for highly sequential tasks unless specialization or separate permissions are essential.
• Add agents only after identifying a coordination benefit that exceeds communication overhead.

Required Output Shapes

Landscape Review

Return:

1. scope and definitions;
2. taxonomy;
3. 8–15 foundational works;
4. mechanism-based map of later work;
5. company and open-source systems;
6. code availability and reproduction shortlist;
7. negative results and open problems;
8. source-linked evidence table;
9. coverage limits and cutoff date.

Paper Deep Dive

Explain the problem, formal setting, agent roles, topology, message/state flow, algorithm, training/inference budget, evaluation, ablations, code, limitations, and what is genuinely multi-agent.

Framework Selection

Return a short recommendation first, then compare control model, durability, async/parallel support, state, observability, approvals, interoperability, provider lock-in, evaluation, license, and migration risk.

Experiment Plan

Use the protocol in evaluation-and-failures.md. Specify hypotheses, topology, stopping rule, message schema, baselines, budget, metrics, datasets, ablations, failure annotation, logging, statistical tests, and reproducibility artifacts.

Evidence Discipline

• Separate peer-reviewed findings, preprints, official product claims, and inference.
• Put citations next to the claims they support.
• State when code exists but does not reproduce every reported experiment.
• Do not call an internal company evaluation a public benchmark.
• Do not infer production readiness from an open-source repository.
• Include important negative evidence such as MAST, conformity studies, and budget-matched debate comparisons.
• Recheck product status and repository maintenance dates because this field changes quickly.

Reference Routing

• Read taxonomy.md for definitions, patterns, and protocol boundaries.
• Read canonical-works.md for the historical and modern paper map.
• Read industry-systems.md for official company systems and claims.
• Read codebases.md for repositories, status, and reproduction fit.
• Read evaluation-and-failures.md for fair comparisons, metrics, logging, and failure analysis.