Adaptive Mas Converter

Adaptive MAS → Skill Converter

Convert a multi-agent system into a single-agent skill. The amount of structure the generated skill retains is determined by the target task's evaluation metric — not by a fixed template.

Core principle: Analyze the task first. The metric determines how much guidance helps versus constrains the agent.

Phase 0: Analyze Task Freedom

Before touching the MAS, read the benchmark's evaluation code. For each metric, assess five dimensions, then estimate a freedom score F ∈ [0, 1].

The Five Dimensions

1. Equivalence strictness — How does the metric judge whether two outputs are equally good?

Read the comparison logic in the evaluation code. The spectrum runs from exact equality operators on one end to continuous scoring functions on the other. Identify where on this spectrum the metric falls.

2. Output space constraints — What must the output satisfy just to be a valid submission?

Some tasks require output in a formal language, or satisfying structural invariants, or choosing from a finite set. Others accept essentially any output and just score it. More constraints mean lower freedom.

3. Near-optimal solution density — How many fundamentally different solutions achieve similar scores?

A qualitative judgment: are good solutions rare and isolated, or are they abundant and diverse? The denser the good region of solution space, the higher the freedom.

4. Perturbation sensitivity — If the output changes slightly, how much does the score move?

On one end, changing a single element can flip the score entirely. On the other, small changes cause proportionally small score changes. Higher sensitivity means lower freedom.

5. Knowledge-to-metric distance — How directly does domain knowledge translate to metric improvement?

Sometimes knowing a domain rule immediately determines whether the output is correct. Sometimes domain intuition provides only vague directional guidance. The more direct the mapping, the more valuable structured knowledge is — and the lower the effective freedom for the agent.

Producing the F Estimate

After analyzing all five dimensions, write a reasoning paragraph covering each, then give F ∈ [0, 1].

• Low F (≈ 0.05–0.25): strict equivalence, high constraints, sparse good solutions, high sensitivity, direct knowledge-to-metric mapping. Structured guidance has maximum value.
• Medium F (≈ 0.25–0.65): partial credit, moderate constraints, moderate density, moderate sensitivity. Knowledge helps but rigid pipelines hurt.
• High F (≈ 0.65–0.95): continuous scoring, low constraints, dense solutions, gradual sensitivity, weak knowledge mapping. The agent's own exploration matters more than prescribed structure.

If the task has multiple metrics, estimate F for each separately. If different datasets use different evaluation logic, note the differences. Report all F values and identify which metric is primary.

Phase 1: Read the MAS

Read the MAS source code, paper, or description. Produce a component inventory.

1.1 Agents

For each agent, record:

• Name and role
• Tools: external computations it calls
• Policy: behavioral instructions (from system prompts, code, or role descriptions)
• Embedded knowledge: domain rules, constraints, or assumptions hidden in prompts or hardcoded logic

1.2 Communication Graph

• Directed edges: which agent sends output to which
• Topological order (if acyclic)
• Cycles (debate, voting, retry loops)

1.3 Identify MAS Patterns

Classify the communication graph into one or more of these patterns. For each, note what it was solving and what to extract:

Critical distinction: There are two layers of "structure" in a MAS:

1. Coordination structure — agent messaging, routing, debate protocols, voting, shared memory. This is a MAS artifact. Always discard, regardless of F.
2. Task decomposition — which subtasks exist, what each produces, quality criteria. This may carry real value. Its retention is F-dependent (Phase 2 decides).

When analyzing a pattern, ask: "Was this architectural choice solving a coordination problem (agents need to talk) or a task problem (the work genuinely decomposes this way)?" Only the latter survives conversion.

1.4 Component Catalog

Classify every extractable component:

1.5 Where to Find Knowledge

Check in this order:

1. System prompts and prompt templates — domain constraints live here
2. Decision trees and routing logic — hardcoded rules are knowledge
3. Docstrings explaining why a check is done
4. Top-level constants, config dicts, lookup tables
5. Paper's methodology section — domain invariants that motivated each component

LLM-as-tool: When a MAS component wraps an LLM call with a domain-expert system prompt, discard the call mechanism but read the prompt for domain constraints, forbidden combinations, and named thresholds. Extract those as knowledge items.

If the MAS has a structured specification (YAML, JSON, or similar config), use it to accelerate the inventory. Otherwise, read the code directly.

Phase 2: Convert

Using the F estimate and the component catalog, decide what to keep, convert, or discard. F is a reference — use judgment.

Always Keep (any F)

Tools — Any component performing external computation the agent cannot do by reasoning alone.

A component is a tool when:

1. It is LLM-independent — same input always gives same output
2. It returns something the agent cannot approximate by thinking
3. Removing it forces the agent to guess

If it could be replaced by a rule the agent reads → knowledge, not a tool.

Never make these into tools: input parsing, explanation generation, output formatting, LLM-based reasoning, self-reflection.

Output format — The schema the calling application expects.

Core factual knowledge — Non-obvious, stable domain facts the agent would not reliably know.

F-Dependent Decisions

Use F as a continuous dial, not a hard threshold. Blend as appropriate.

Domain Knowledge

Pipeline / Workflow

Empirical Heuristics

Decision Logic / Routing

Always Discard (any F)

• Agent-to-agent coordination and messaging protocols
• Voting, debate, and self-reflection mechanisms
• Retry-and-route loops
• LLM-based input parsers and output formatters

Mixed-F Tasks

When a task has multiple metrics with different F values:

1. Set the overall style based on the primary metric's F
2. For secondary metrics with substantially lower F, add targeted knowledge items for those aspects
3. The agent should adapt granularity within a single skill — more structure for the rigid parts, less for the flexible parts

Phase 3: Write the Skill

Tool Files

Write tools in a tools/ directory.

"""tools/<module>.py — <one-line description>"""
import <imports>


def <function_name>(param_a: str, param_b: float) -> dict:
    """
    <One-line description.>
    Returns: {<field>: <description>, error: str or None}
    """
    try:
        return {"<field>": ..., "error": None}
    except Exception as e:
        return {"error": str(e)}

Rules:

• Inputs: JSON-serializable only
• Outputs: flat dict with diagnostics
• All imports at top of file
• Wrap body in try/except
• No LLM calls inside tools

After each tool, verify:

python -c "import sys; sys.path.insert(0, '<SKILL_DIR>'); from tools.<module> import <fn>; print('OK')"

Generated SKILL.md Structure

The output skill must use these standard sections. Which sections appear and how detailed they are depends on F.

Section Reference

Frontmatter (always):

---
name: <skill-name>
description: <when to invoke and what task it addresses>
---

What This Skill Does (always): Task description, expected inputs and outputs — one paragraph.

Domain Knowledge (low and mid F):

• Only write knowledge NOT already encoded in a tool's output or docstring.
• Pattern: [conditional: <condition>] <knowledge item> or [always] <rule>.
• No thresholds a tool already returns. No step sequences.

Available Tools (always when tools exist): For each tool, document:

• What it computes (one sentence)
• When to use (specific condition the agent can check before calling)
• What the output tells you (field-by-field: what the value means, what to do next)
• What to do if it errors (fallback)

Tool Invocation (always when tools exist):

python - <<'PYEOF'
import sys, json
sys.path.insert(0, "<SKILL_DIR>")
from tools.<module> import <function_name>

result = <function_name>("/path/to/data", ...)
print(json.dumps(result, default=str))
PYEOF

Recommended Workflow (low F only): Phases with handover contracts (output fields + format). Use "suggested" not "must."

Dependencies (always): pip install line for required packages.

F-Dependent Assembly

Length discipline: If the generated SKILL.md exceeds the target length, for each line over the limit ask: "Is this already returned by a tool's output field or docstring?" If yes, delete it — the tool owns it. "Could a capable agent derive this without being told?" If yes, delete it.

SKILL.md describes when to call tools and what to do with results. Tools explain how and what the numbers mean. Do not duplicate between them.

Quality Checks

Before finalizing, run these checks on the generated skill:

Pipeline leak — Are there step sequences, numbered workflows, or box-arrow diagrams? Appropriate only when F is low. Otherwise, convert each step to a conditional knowledge item or delete.

Innovation tax — For every knowledge item, ask: "Does this restrict the agent from discovering a better approach?" The higher F is, the more aggressively you should cut.

Tone test — Does each item read as a reference (stating a fact about the world) or an instruction (telling the agent what to do)? References are safe. Instructions carry risk. The higher F is, the more instructions should become references or get cut.

Leanness — Read the skill as if seeing it for the first time. Cut every sentence that does not give new information the agent could not derive itself.

Tool triggers — Every tool must have a realistic trigger condition. A tool that never gets called is dead weight.

Phase 4: Validate and Calibrate

Structural Validation

Check the generated skill package:

• SKILL.md has frontmatter with name and description
• Every required section for the chosen F level is present
• No step sequences appear outside of a low-F workflow section (look for "Step 1:", "First, then, next, finally" patterns)
• Every knowledge item uses reference tone, not instruction tone (unless F is very low)
• Every tool file passes the import check from Phase 3
• All tool function signatures match the documentation in SKILL.md

Quality Review Against F

• F was low: Does the skill provide enough guidance for an agent unfamiliar with this domain to produce correct output? Are handover contracts specific enough?
• F was high: Would a capable agent feel unnecessarily constrained by anything in the skill? If yes, cut it.
• Any F: Are all tools load-bearing? Is all knowledge non-obvious? Is the output format correct? Are all MAS coordination mechanisms discarded?

Generate Evaluation Cases

Create evals/test_cases.json with 3–5 test cases covering:

Tool trigger test — A prompt where a specific tool's trigger condition is clearly met. Expected: the tool is called.

Tool skip test — A prompt where no tool condition is met (e.g., the result is already provided for interpretation). Expected: no tools are called.

Boundary test — A prompt where the trigger condition is ambiguous or partially met. Expected: the agent flags the ambiguity before acting.

Each test case records:

• id, description, prompt
• expected_tool_triggers: tool names that should fire
• expected_tool_skips: tool names that must not fire
• expected_knowledge_applied: key concepts that should appear in reasoning

Calibration Check

For each test case, mentally simulate what the agent would do when given the prompt plus the generated skill. Verify:

1. Does the tool trigger description in the skill unambiguously match the test prompt's conditions?
2. Could the agent reasonably infer the correct action from the skill's knowledge section?
3. Is there any knowledge item that would mislead the agent on this test case?

If a test case reveals a mismatch, revise the skill before finalizing.

Quick Reference: Component Disposition

Output

<skill-name>/
├── SKILL.md           ← generated skill (structure varies with F)
├── tools/
│   ├── <module_a>.py
│   └── <module_b>.py
└── evals/
    └── test_cases.json  ← trigger/skip/boundary tests