Twilio Extract

Twilio Extract

Read-side tool: pulls data OUT of Twilio (conversations, messages, participants, Studio Flow executions/steps, debugger alerts) via the REST API. Also exposes two opt-in mutating commands (update-attributes, close) gated behind --yes, for the same debugging use cases documented in this org's Twilio integration docs.

When to Activate

• "busca las conversaciones de Twilio pa este número"
• "qué attributes tiene esta conversación de Twilio"
• "revisa la ejecución del Studio Flow pa este execution sid"
• "hay alertas recientes en el debugger de Twilio"
• Cross-referencing a Twilio conversation with our own DB / Langfuse session

Credentials — .env convention

One .env file per account/workspace at ~/.twilio_<workspace>.env — same pattern already in use in this environment (~/.twilio_qa.env, ~/.twilio_qa_prod.env), never committed to git.

Template: .env.example in this skill's directory. Required vars:

WORKSPACE_NAME=       # label for logs
ACCOUNT_SID=          # AC...
AUTH_TOKEN=           # live Auth Token (Test SID/Token don't work for Conversations/Studio)

Var names are bare ACCOUNT_SID/AUTH_TOKEN (not TWILIO_*) to match the existing files. The export prefix is optional — python-dotenv strips it.

Auth is HTTP Basic: username=Account SID, password=Auth Token. Set chmod 600 on the env file.

Tools in this skill

• scripts/twilio_client.py — reusable TwilioClient class (auth, cursor pagination via meta.next_page_url, retry on 429/5xx). Import this if writing a custom script.
• scripts/extract.py — CLI wrapper. Covers the common cases without writing code.

CLI usage

cd ~/.claude/skills/twilio-extract/scripts
uv run extract.py <kind> --env-file ~/.twilio_qa_prod.env [options]

<kind>: conversations-by-address | conversation | messages | participants | webhooks | update-attributes | close | studio-flow | studio-executions | studio-steps | debugger-alerts | health

# Every conversation a WhatsApp number has participated in (most common query)
uv run extract.py conversations-by-address --env-file ~/.twilio_qa_prod.env \
    --address "whatsapp:+584149190631"

# Single conversation (state, attributes, timers)
uv run extract.py conversation --env-file ~/.twilio_qa_prod.env --sid CH68bdf4bfe0d3404e9ef707220eb78f7e

# Messages in a conversation, oldest first
uv run extract.py messages --env-file ~/.twilio_qa_prod.env \
    --sid CH68bdf4bfe0d3404e9ef707220eb78f7e --order asc

# Scoped webhooks attached to a conversation (how our adapter gets onMessageAdded)
uv run extract.py webhooks --env-file ~/.twilio_qa_prod.env --sid CH68bdf4bfe0d3404e9ef707220eb78f7e

# Studio Flow: recent executions, then the steps of one execution
uv run extract.py studio-executions --env-file ~/.twilio_qa.env --flow-sid FW17995da90888cd3be079dd26dc04d820
uv run extract.py studio-steps --env-file ~/.twilio_qa.env \
    --flow-sid FW17995da90888cd3be079dd26dc04d820 --execution-sid FNxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

# Debugger alerts (errors/warnings Twilio raised recently)
uv run extract.py debugger-alerts --env-file ~/.twilio_qa_prod.env

# Sanity check credentials before a real pull
uv run extract.py health --env-file ~/.twilio_qa_prod.env

# Mutating (opt-in, requires --yes): force-close to make the next inbound re-run the Flow
uv run extract.py close --env-file ~/.twilio_qa.env --sid CHxxxx --yes

# Mutating (opt-in, requires --yes): set attributes — REPLACES the whole blob,
# so read `conversation` first and merge client-side if you need to keep existing keys
uv run extract.py update-attributes --env-file ~/.twilio_qa_prod.env --sid CHxxxx \
    --json '{"langfuse_session_url":"https://us.cloud.langfuse.com/project/.../sessions/..."}' --yes

Output: --output/-o file --format/-f json|jsonl|csv (default: prints JSON to stdout). csv flattens nested fields (attributes/messaging binding/etc.) to JSON strings per cell.

--max-items caps total items across pages for list calls (conversations-by-address, messages, participants, studio-executions, studio-steps, debugger-alerts) — Twilio's list resources don't return a total count up front, so an unscoped pull can page for a long time on a busy account; the cap is a safety net, not a substitute for narrowing the query (there's no server-side date filter on ParticipantConversations, so narrow by address instead).

API reference (what the client wraps)

Full reference: https://www.twilio.com/docs/conversations/api, https://www.twilio.com/docs/studio/rest-api

Gotchas

• Address format matters. ParticipantConversations?Address= needs the full channel-prefixed address — whatsapp:+584149190631 for WhatsApp, bare +584149190631 only for plain SMS/voice. Forgetting the whatsapp: prefix returns an empty list, not an error — looks like "no conversations" when really it's a format mismatch.
• Attributes/State writes REPLACE, they don't merge. POST /Conversations/{sid} with Attributes=... overwrites the whole JSON blob server-side. To add one key without losing others: conversation (GET) first, merge client-side, then update-attributes with the full merged JSON. Same contract the app's own AsyncTwilioClient.update_conversation follows (backend/cheo/channels/twilio/client.py).
• Test SID + Test Token don't work here. They only simulate REST responses for Conversations API and webhooks — always use the live Account SID + Auth Token.
• Studio Flow executions ≠ conversations. An execution is one run of the flow triggered by an inbound message; studio-steps shows which widget matched (e.g. a whitelist split-based-on widget routing to a bot vs. a human queue) — the fastest way to answer "why didn't this number get routed correctly."
• No server-side date filter on ParticipantConversations. It returns everything for that address, oldest first typically unbounded — use --max-items and inspect conversation_date_created client-side if you only care about a recent window.
• Rate limits / retries. TwilioClient retries 429 (honoring Retry-After) and 5xx with backoff, same as the Langfuse extractor. Don't lower retries for bulk pulls across many addresses.

Cross-referencing with ai-platform's own data (Langfuse / Postgres)

This org's Twilio channel (backend/cheo/channels/twilio/) links every Twilio conversation to an internal Conversation row:

• Conversation.external_id (Postgres) = the Twilio Conversation SID (CH...)
• Conversation.id (Postgres, our internal UUID) = the Langfuse session_id (build_session_url in cheo/integrations/langfuse/client.py builds the deep-link from str(conversation.id) directly — no separate mapping table)

So to go from a phone number to its Langfuse trace:

1. conversations-by-address --address whatsapp:+<number> -> get the Twilio SID(s)
2. If the conversation is recent, its conversation_attributes may already carry langfuse_session_url (set by twilio/adapter.py on turn completion) — done.
3. Otherwise, look up Conversation.id in Postgres by external_id IN (<SIDs>) and build the URL yourself: {LANGFUSE_BASE_URL}/project/{LANGFUSE_PROJECT_ID}/sessions/{id}. Langfuse tracing runs for every conversation regardless of whether the attribute got persisted — the attribute is a convenience deep-link, not a gate on the session existing.
4. A Twilio conversation with zero matching Postgres row usually means it was never routed to Cheo at all (e.g. the provider's Studio Flow whitelist didn't match that number) — not a bug on our side to chase in the DB.

Use the langfuse-extract skill's session/debug-session kinds once you have the internal id, to pull the actual trace content.

Reference Skills

• Pulling the Langfuse session/traces once you have the internal conversation id → skill: langfuse-extract
• Adding/changing Twilio channel logic in the app → not this skill, it's a coding task