Integrate Openclaw

Integrate Patter with OpenClaw (brain on the line)

OpenClaw is a self-hosted gateway of LLM-backed agents with scoped tool connections (calendars, customer DBs, …). Patter is the voice layer: it owns the carrier leg, talks with low latency, and reaches into OpenClaw only when a turn needs real data or an action. This is the brain-on-the-line architecture — the inverse of wiring OpenClaw in as a "Custom LLM" that sits on every turn (the topology that kills calls on slow tools).

CALLER (any number, no allowlist)
   │
   ▼
Twilio / Telnyx DID  ──►  PATTER voice agent (OpenAI Realtime, low-latency EN)
                              │  owns turn-taking + TTS locally
                              │  speaks "let me check, one moment"
                              ▼
                          consult ──► local adapter ──► OpenClaw
                              ▲          (loopback)        /v1/chat/completions
                              │                            model="openclaw/<receptionistAgentId>"
                          speaks the reply                 (ONE scoped agent)

The two directions — and which one to use

They compose, but for an inbound receptionist line Direction B is the one to build first. The rest of this skill is mostly Direction B; Direction A is at the end.

Why not OpenClaw's native voice plugin?

OpenClaw's built-in voice plugin only accepts inbound calls from an allowlist (inboundPolicy: "allowlist" + allowFrom: [...]); there is no "accept anyone" mode. That is useless for a public after-hours line where random people call. Patter owns the DID and answers everyone, then reaches the receptionist over OpenClaw's operator-side /v1/chat/completions gateway — the caller is never an OpenClaw "sender" subject to the allowlist. This is the single clearest reason to put Patter in front.

Direction B — Patter consults OpenClaw mid-call

Step 1 — Enable OpenClaw's chat-completions gateway

OpenClaw exposes an OpenAI-compatible POST /v1/chat/completions. It is disabled by default. Enable it in ~/.openclaw/openclaw.json (JSON5):

{
  gateway: {
    http: {
      endpoints: {
        chatCompletions: {
          enabled: true,
        },
      },
    },
  },
}

1. A dedicated least-privileged receptionist agent with a tight per-agent tools.allow / tools.deny (see Step 5).
2. One gateway per client — bind the gateway to loopback / tailnet only and never expose /v1/chat/completions to the public internet. On a single always-on Mac Mini per client (one OS user = one gateway = one credential set), this isolation is free.
3. An auth credential on the gateway (mirror OpenClaw's own gateway-auth guidance — do not stand up an unauthenticated, network-reachable endpoint).

Confirm the exact gateway bind/auth keys and the model-routing alias forms against the live OpenClaw docs (docs.openclaw.ai) before you ship — they are OpenClaw-side config, not Patter's.

Step 2 — Route the consult to ONE specific receptionist agent

On /v1/chat/completions the model field is an agent target:

• "openclaw" / "openclaw/default" → the default agent — which may be the master / CEO / financial agent and can drift between environments. Never use this from the voice layer.
• "openclaw/<receptionistAgentId>" → one explicit, named agent. Always pin this. (Alias forms openclaw:<id> / agent:<id> are commonly accepted — verify against OpenClaw docs.)

Per client, set the agentId explicitly:

Step 3 — Point consult at the OpenClaw agent

Recommended: the native target — no adapter. Patter's consult speaks OpenClaw's /chat/completions endpoint directly; point it at one scoped agent in a single line:

from getpatter import ConsultConfig, OpenAIRealtime2

agent = phone.agent(
    engine=OpenAIRealtime2(),
    system_prompt="You are the after-hours receptionist...",
    consult=ConsultConfig.openclaw("receptionist"),
)

import { openclawConsult, OpenAIRealtime2 } from "getpatter";

const agent = phone.agent({
  engine: new OpenAIRealtime2(),
  systemPrompt: "You are the after-hours receptionist...",
  consult: openclawConsult("receptionist"),
});

It targets model="openclaw/receptionist", sends the call id as the OpenAI user field plus the x-openclaw-session-key header (one OpenClaw session per call), reads the operator-grade bearer from OPENCLAW_API_KEY (never logged), auto-enables allow_loopback for the co-located gateway, and attaches a default "let me check" reassurance filler. Notify OpenClaw at call end with openclaw_post_call_notifier("receptionist") / openclawPostCallNotifier("receptionist") on serve(on_call_end=...) to post the call record (caller, line, duration, transcript) to the same agent and session.

Escape hatch — a hand-written adapter, only when you need a custom request/response mapping (e.g. a non-OpenClaw back office). Patter POSTs { request, call_id, caller, callee }; the adapter must:

1. Target one named agent (model="openclaw/<receptionistAgentId>").
2. Carry call_id as the OpenClaw session key so a multi-turn call maps to one OpenClaw session (continuity), and forward caller for caller-ID prefetch of the customer record.
3. Return a concise, spoken-ready string (don't dump a raw API blob back into the voice context).

# adapter.py — forwards Patter consult requests to ONE OpenClaw receptionist agent
import os
import httpx
from fastapi import FastAPI, Request

app = FastAPI()

OPENCLAW_URL = os.environ["OPENCLAW_URL"]          # e.g. http://127.0.0.1:18789/v1/chat/completions
OPENCLAW_TOKEN = os.environ["OPENCLAW_GATEWAY_TOKEN"]
RECEPTIONIST_AGENT = os.environ.get("OPENCLAW_AGENT", "openclaw/receptionist")

@app.post("/consult")
async def consult(req: Request):
    body = await req.json()
    request_text = body.get("request", "")
    call_id = body.get("call_id", "")
    caller = body.get("caller", "")

    payload = {
        "model": RECEPTIONIST_AGENT,                # ONE scoped agent, never "openclaw"
        "user": call_id,                            # one OpenClaw session per call
        "messages": [{
            "role": "user",
            # caller-id is forwarded so the brain can prefetch the customer record
            "content": f"[caller={caller}] {request_text}",
        }],
    }
    headers = {"Authorization": f"Bearer {OPENCLAW_TOKEN}"}

    # 75 s is sized to OpenClaw's worst-case tool time (see "timeout ladder").
    async with httpx.AsyncClient(timeout=75.0) as client:
        resp = await client.post(OPENCLAW_URL, json=payload, headers=headers)
        resp.raise_for_status()
        content = resp.json()["choices"][0]["message"]["content"]

    # Trim to a short, spoken-ready reply before it re-enters the voice context.
    return {"reply": content.strip()[:600]}

// adapter.ts — forwards Patter consult requests to ONE OpenClaw receptionist agent
import express from "express";

const app = express();
app.use(express.json());

const OPENCLAW_URL = process.env.OPENCLAW_URL!;            // http://127.0.0.1:18789/v1/chat/completions
const OPENCLAW_TOKEN = process.env.OPENCLAW_GATEWAY_TOKEN!;
const RECEPTIONIST_AGENT = process.env.OPENCLAW_AGENT ?? "openclaw/receptionist";

app.post("/consult", async (req, res) => {
  const { request = "", call_id = "", caller = "" } = req.body ?? {};

  const payload = {
    model: RECEPTIONIST_AGENT,                              // ONE scoped agent, never "openclaw"
    user: call_id,                                          // one OpenClaw session per call
    messages: [{ role: "user", content: `[caller=${caller}] ${request}` }],
  };

  const resp = await fetch(OPENCLAW_URL, {
    method: "POST",
    headers: { "Content-Type": "application/json", Authorization: `Bearer ${OPENCLAW_TOKEN}` },
    body: JSON.stringify(payload),
    signal: AbortSignal.timeout(75_000),                    // see "timeout ladder"
  });
  const data = await resp.json();
  const content: string = data.choices[0].message.content;

  res.json({ reply: content.trim().slice(0, 600) });        // short, spoken-ready
});

app.listen(8000);

The adapter is also where you can drop slow work onto an OpenClaw sub-agent (e.g. spawn a worker, return a short "checking…" and let a later turn fetch the result). Optional — the in-band reassurance below already covers dead air for a single 60 s lookup.

Step 4 — Point the Patter agent at the adapter

ConsultConfig (Python) / consult (TS) auto-injects a consult_agent tool the in-call agent can call. On a single Mac Mini the adapter is on loopback, so opt in with allow_loopback / allowLoopback.

from getpatter import Patter, Twilio, OpenAIRealtime2, ConsultConfig

phone = Patter(carrier=Twilio(), phone_number="+15550001234")

agent = phone.agent(
    engine=OpenAIRealtime2(),                    # low-latency English, strongest tool flow
    system_prompt=(
        "You are the after-hours receptionist for Acme Roofing. "
        "When the caller asks about appointments, availability, or account "
        "details, call `consult_agent` with a clear request. "
        "Say a brief 'let me check' first, then read back the answer."
    ),
    first_message="Thanks for calling Acme Roofing, how can I help?",
    consult=ConsultConfig(
        url="http://127.0.0.1:8000/consult",
        timeout_s=75.0,                          # sized to the brain's worst-case tool time
        allow_loopback=True,                     # adapter is co-located on this box
        headers={"Authorization": "Bearer <adapter-token>"},  # optional, never logged
        # tool_name / description default to "consult_agent" + a sensible prompt;
        # override description to steer WHEN the agent escalates.
    ),
)

phone.serve(agent)

import { Patter, Twilio, OpenAIRealtime2 } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });

const agent = phone.agent({
  engine: new OpenAIRealtime2(),                 // low-latency English, strongest tool flow
  systemPrompt:
    "You are the after-hours receptionist for Acme Roofing. " +
    "When the caller asks about appointments, availability, or account " +
    "details, call consult_agent with a clear request. " +
    "Say a brief 'let me check' first, then read back the answer.",
  firstMessage: "Thanks for calling Acme Roofing, how can I help?",
  consult: {
    url: "http://127.0.0.1:8000/consult",
    timeoutMs: 75_000,                           // sized to the brain's worst-case tool time
    allowLoopback: true,                         // adapter is co-located on this box
    headers: { Authorization: "Bearer <adapter-token>" }, // optional, never logged
  },
});

await phone.serve({ agent });

Defaults to know: consult timeout defaults to 30 s (timeout_s=30.0 / timeoutMs: 30000), the tool is named consult_agent, and consult is injected in Realtime and Pipeline only (a warning is emitted if you set it with ElevenLabs ConvAI).

Long-tool-call survival (the literal blocker)

The prior failure with another platform was: the agent makes a 30-60 s tool call (sometimes browser automation) and the call dies. The fix is three layers, all of which must hold.

1. The timeout ladder — the call dies at the SHORTEST link

Verify the per-server timeout is honoured. OpenClaw's docs don't publish a default for the per-server MCP timeout, and an MCP client can impose its own read-timeout ceiling that silently caps a long blocking call regardless of config. After wiring, probe the server with openclaw mcp doctor patter --probe (a live connection check). openclaw mcp status --verbose shows the resolved timeout but does NOT open a connection.

2. Reassurance — speak a filler the instant the tool starts

Without a filler the caller hears dead air for the whole lookup and assumes the line froze. Attach reassurance to the consult tool (or any slow tool) so the agent speaks immediately and keeps the media stream alive while the brain works.

The consult_agent tool is auto-built, so to give it reassurance the cleanest path today is to add your own slow tool that wraps the consult, OR steer the agent via the system prompt to say "let me check, one moment" before it calls consult_agent. For a hand-built slow tool, set reassurance on the Tool (Python) / ToolDefinition (TS):

from getpatter import Tool

# reassurance is a field on the Tool dataclass (string shorthand → after_ms=1500,
# or a dict {"message": str, "after_ms": int}). Realtime mode only today.
check_schedule = Tool(
    name="check_schedule",
    description="Look up the caller's appointment in the calendar.",
    parameters={"type": "object", "properties": {"day": {"type": "string"}}, "required": ["day"]},
    handler=my_slow_handler,                         # may take 30-60 s
    reassurance="Let me check the calendar for you, one moment.",
)

// In TS, reassurance and a per-tool timeoutMs are fields on the raw
// ToolDefinition (defineTool does not expose them — build the object directly).
const checkSchedule = {
  name: "check_schedule",
  description: "Look up the caller's appointment in the calendar.",
  parameters: {
    type: "object",
    properties: { day: { type: "string" } },
    required: ["day"],
  },
  handler: mySlowHandler,                            // may take 30-60 s
  reassurance: "Let me check the calendar for you, one moment.",
  timeoutMs: 60_000,                                 // raise off the 10 s default
};

3. Per-tool timeout

• TypeScript: ToolDefinition.timeoutMs (default 10 000 ms, clamped to 300 000 ms). Raise it to 60_000 for slow tools — shown above. A timeout returns { error, fallback: true } and is not retried.
• Python: timeout_s on the tool() factory / Tool dataclass since 0.6.4 (tool(name=..., handler=..., timeout_s=60.0); default None keeps the 10 s behaviour, clamped to 300 s). Same terminal semantics as TS. The consult path keeps its own independent ConsultConfig.timeout_s.

Why this beats "OpenClaw as a Custom LLM"

The failure mode to avoid is wiring OpenClaw in as the per-turn LLM (a "Custom LLM" / /v1/chat/completions on the critical path of every utterance). Then the brain's full reasoning + tool latency blocks every response, and a 60 s tool stalls the turn with no acknowledge-then-continue primitive. Patter's consult is the inverse: the local voice model owns turn-taking and TTS, and the brain is a consulted specialist invoked only on hard turns. Keep it that way.

Noise & turn-detection for speakerphone

Contractors call from job sites on speakerphone. Tiny noises (a mouse move, the phone shifting) can false-trigger turn detection and cut the agent off mid-sentence. The live levers:

Realtime receptionist (reassurance works here — preferred for this line):

from getpatter import Patter, Twilio, OpenAIRealtime2, RealtimeTurnDetection

phone = Patter(carrier=Twilio(), phone_number="+15550001234")
agent = phone.agent(
    engine=OpenAIRealtime2(),
    system_prompt="You are the after-hours receptionist for Acme Roofing.",
    first_message="Thanks for calling Acme Roofing, how can I help?",
    barge_in_threshold_ms=500,                       # noisy speakerphone
    openai_realtime_noise_reduction="far_field",     # speakerphone preset
    realtime_turn_detection=RealtimeTurnDetection(
        type="server_vad", threshold=0.7, silence_duration_ms=700,
    ),
    consult=ConsultConfig(url="http://127.0.0.1:8000/consult", timeout_s=75.0, allow_loopback=True),
)

import { Patter, Twilio, OpenAIRealtime2 } from "getpatter";

const phone = new Patter({ carrier: new Twilio(), phoneNumber: "+15550001234" });
const agent = phone.agent({
  engine: new OpenAIRealtime2(),
  systemPrompt: "You are the after-hours receptionist for Acme Roofing.",
  firstMessage: "Thanks for calling Acme Roofing, how can I help?",
  bargeInThresholdMs: 500,                           // noisy speakerphone
  openaiRealtimeNoiseReduction: "far_field",         // speakerphone preset
  realtimeTurnDetection: { type: "server_vad", threshold: 0.7, silenceDurationMs: 700 },
  consult: { url: "http://127.0.0.1:8000/consult", timeoutMs: 75_000, allowLoopback: true },
});

Pipeline variant — deeper audio chain when you control the STT/LLM/TTS stack (see build-voice-agent → references/pipeline-mode.md for the full lever docs):

from getpatter import Patter, Twilio, NamoTurnDetector
from getpatter.providers.silero_vad import SileroVAD

phone = Patter(carrier=Twilio(), phone_number="+15550001234")
agent = phone.agent(
    # Pipeline mode (no engine=) — exposes the full inbound audio chain
    system_prompt="You are the after-hours receptionist for Acme Roofing.",
    first_message="Thanks for calling Acme Roofing, how can I help?",
    barge_in_threshold_ms=500,
    denoiser="krisp-viva-tel-v2",                    # BYO Krisp license
    high_pass_hz=100,                                # hum + handling rumble
    agc=True,                                        # quiet talkers
    vad=SileroVAD.load(activation_threshold=0.8, min_silence_duration=0.7),
    turn_detector=NamoTurnDetector.load(),           # PATTER_NAMO_MODEL
    consult=ConsultConfig(url="http://127.0.0.1:8000/consult", timeout_s=75.0, allow_loopback=True),
)

Inbound: anyone can call

Patter answers any caller on its DID — no allowlist. There are two ways to set the inbound agent:

• In the SDK: the agent you pass to phone.serve(agent) answers all inbound calls. Verify X-Twilio-Signature / Telnyx Ed25519 at the carrier edge (Patter does this for you) — that is the only trust boundary the public touches.
• Via patter-mcp (Direction A): the configure_inbound tool sets the default agent that answers all future inbound calls. Fields: systemPrompt, firstMessage (opt.), engineMode (default pipeline), sttProvider / llmProvider / ttsProvider (pipeline), voice, language.

Contrast with OpenClaw's native voice plugin, whose inboundPolicy is allowlist-only (disabled by default) — it structurally cannot front random after-hours callers. Patter terminates the carrier leg for everyone; OpenClaw stays the brain behind it.

Least-privilege agent scoping (OpenClaw side)

The voice layer must reach only the receptionist, never the master agent. Model the receptionist as a top-level agents.list[] entry (not a sub-agent of the master — sub-agent auth can fall back to the parent's credentials), with:

• its own workspace / agentDir / auth profiles,
• a tight tools.allow (only the calendar + customer-DB tools) and explicit tools.deny (exec / write / browser / gateway),
• per-agent sandbox (mode: all, scope: agent).

Expose any third-party MCP tools (calendar, customer DB) only to that agent (per-server agent projection + a narrow toolFilter.include). Per-client acceptance gate before going live:

1. List agents and their bindings; confirm the receptionist is bound to the inbound path and the master is not reachable from it.
2. Inspect the receptionist's effective tool list — it CAN reach the calendar / DB tools, and CANNOT reach exec / financial tools.
3. Run OpenClaw's security audit; verify the gateway binds loopback (not public).
4. Run one full long-tool-call end to end (a 45-60 s lookup) and confirm the call survives and the agent speaks the result.

The exact OpenClaw config keys, CLI verbs, and audit commands are OpenClaw-side — confirm them against docs.openclaw.ai. Patter's contract is only the consult HTTP body and the named-agent model value.

Direction A — OpenClaw places calls via Patter (patter-mcp)

Use this when OpenClaw should initiate calls. Run the MCP server from a local clone (it is not on npm) and register it in ~/.openclaw/openclaw.json. Prefer a long-lived streamable-http server on an always-on box — OpenClaw reaps idle stdio runtimes, which would re-spawn the embedded Patter server per session.

{
  mcp: {
    servers: {
      patter: {
        url: "http://localhost:3000/mcp",
        transport: "streamable-http",
        timeout: 600,                          // blocking make_call runs the whole call
        toolFilter: {
          // Narrow to the phone path — drop configure_inbound / get_metrics here.
          include: ["make_call", "call_third_party", "get_transcript", "get_calls", "end_call"],
        },
        // Scope this server to the receptionist via the agent's per-agent tool
        // policy (agents.list[].tools.allow + sandbox alsoAllow of "bundle-mcp").
        // NOTE: mcp.servers.<name>.codex.agents projects ONLY to the Codex
        // app-server runtime, NOT embedded-OpenClaw agents — don't rely on it here.
      },
    },
  },
  tools: {
    sandbox: {
      tools: {
        // MCP tools live in the "bundle-mcp" group; allowlist it (or "patter__*").
        alsoAllow: ["bundle-mcp"],
      },
    },
  },
}

The seven patter-mcp tools: make_call, call_third_party, get_calls, get_transcript, end_call, get_metrics, configure_inbound. With wait: true, make_call / call_third_party block until the call ends and return the outcome plus transcript in one response.

Prefer OAuth / mTLS over static env-var secrets committed in the config, and verify reachability after wiring (openclaw mcp doctor patter --probe).

Gotchas

• Never set model: "openclaw" / "openclaw/default" from the voice layer. It resolves to the default agent (possibly the master) and drifts between environments. Always pin "openclaw/<receptionistAgentId>".
• The gateway credential is operator-grade. The model field selects a persona, not a permission scope. Isolation = least-privileged agent + loopback binding + one-gateway-per-client, not the token.
• Consult timeout defaults to 30 s — too short for 30-60 s automation. Set timeout_s / timeoutMs to 75-90 s AND raise the OpenClaw gateway timeout. The call dies at the shortest link.
• Reassurance is Realtime-only. Pipeline ignores it. Prefer Realtime for the receptionist line so the "let me check" filler actually plays.
• Raise the per-tool timeout on slow tools — both SDKs default to 10 s. Since 0.6.4: Python tool(..., timeout_s=60.0), TS ToolDefinition.timeoutMs: 60_000 (both clamped to 300 s; a timeout is terminal, not retried).
• allow_loopback is the intended shape here, not a hack — on a co-located Mac Mini the adapter and gateway are on loopback. It relaxes only the consult URL's host check; non-HTTP(S) schemes are still rejected.
• The adapter must forward call_id / caller. The default doc adapter drops them; without user=call_id every consult starts a fresh OpenClaw session (no continuity) and caller-ID prefetch is impossible.
• echo_cancellation is a no-op on PSTN calls — Patter's AEC only applies to browser/native audio; the carrier handles line echo. The real noise levers are openai_realtime_noise_reduction="far_field" + realtime_turn_detection (Realtime, 0.6.4+) and denoiser / high_pass_hz / agc / Silero VAD / turn_detector (Pipeline).
• patter-mcp is not on npm. Run it from a local clone of PatterAI/patter-mcp.

Common errors

Related skills

• setup-patter — install the SDK and provision provider/carrier keys first.
• build-voice-agent — define the in-call agent that does the consulting.
• add-tools-and-handoffs — custom tools, reassurance, guardrails, transfer_call / end_call.
• configure-telephony — wire the Twilio / Telnyx inbound webhook.
• inspect-calls-and-metrics — confirm consult latency and call outcomes.

References

• Patter OpenClaw integration: https://docs.getpatter.com/integrations/openclaw
• Patter consult feature: https://docs.getpatter.com/python-sdk/consult · https://docs.getpatter.com/typescript-sdk/consult
• Patter tools (reassurance, per-tool timeout): https://docs.getpatter.com/python-sdk/tools · https://docs.getpatter.com/typescript-sdk/tools
• Patter MCP server: https://github.com/PatterAI/patter-mcp
• OpenClaw docs (gateway, agents, MCP): https://docs.openclaw.ai