How do I install the Phone Agent skill?

Run npx skillmds@latest add delorenj/phone-agent in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Phone Agent safe to use?

This skill has not completed SkillMD's automated safety review yet. Independent scanners report: SkillSpector: PASS, Skill Scanner: PASS. Capability flags: executes scripts, makes network calls. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Phone Agent?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Phone Agent free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Phone Agent?

delorenj (@delorenj) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Phone Agent by delorenj — Agent Skill

Name: Phone Agent
Rating: 4.666666666666667 (3 reviews)
Author: delorenj

Phone Agent

Run a real-time AI phone agent using Twilio, Deepgram, and ElevenLabs. Handles incoming calls, transcribes audio, generates responses via LLM, and speaks back via streaming TTS. Use when user wants to: (1) Test voice AI capabilities, (2) Handle phone calls programmatically, (3) Build a conversational voice bot.

delorenj Updated Sep 10, 2026 1 repo stars

File contents

Phone Agent Skill

Runs a local FastAPI server that acts as a real-time voice bridge.

Architecture

Twilio (Phone) <--> WebSocket (Audio) <--> [Local Server] <--> Deepgram (STT)
                                                  |
                                                  +--> OpenAI (LLM)
                                                  +--> ElevenLabs (TTS)

Prerequisites

Twilio Account: Phone number + TwiML App.
Deepgram API Key: For fast speech-to-text.
OpenAI API Key: For the conversation logic.
ElevenLabs API Key: For realistic text-to-speech.
Ngrok (or similar): To expose your local port 8080 to Twilio.

Setup

Install Dependencies:

pip install -r scripts/requirements.txt

Set Environment Variables (in ~/.moltbot/.env, ~/.clawdbot/.env, or export):

export DEEPGRAM_API_KEY="your_key"
export OPENAI_API_KEY="your_key"
export ELEVENLABS_API_KEY="your_key"
export TWILIO_ACCOUNT_SID="your_sid"
export TWILIO_AUTH_TOKEN="your_token"
export PORT=8080

Start the Server:
```
python3 scripts/server.py
```
Expose to Internet:
```
ngrok http 8080
```
Configure Twilio:
- Go to your Phone Number settings.
- Set "Voice & Fax" -> "A Call Comes In" to Webhook.
- URL: https://<your-ngrok-url>.ngrok.io/incoming
- Method: POST

Usage

Call your Twilio number. The agent should answer, transcribe your speech, think, and reply in a natural voice.

Customization

System Prompt: Edit SYSTEM_PROMPT in scripts/server.py to change the persona.
Voice: Change ELEVENLABS_VOICE_ID to use different voices.
Model: Switch gpt-4o-mini to gpt-4 for smarter (but slower) responses.

---name: phone-agentdescription: Run a real-time AI phone agent using Twilio, Deepgram, and ElevenLabs. Handles incoming calls, transcribes audio, generates responses via LLM, and speaks back via streaming TTS. Use when user wants to: (1) Test voice AI capabilities, (2) Handle phone calls programmatically, (3) Build a conversational voice bot.---# Phone Agent SkillRuns a local FastAPI server that acts as a real-time voice bridge.## Architecture```Twilio (Phone) <--> WebSocket (Audio) <--> [Local Server] <--> Deepgram (STT) | +--> OpenAI (LLM) +--> ElevenLabs (TTS)```## Prerequisites1. **Twilio Account**: Phone number + TwiML App.2. **Deepgram API Key**: For fast speech-to-text.3. **OpenAI API Key**: For the conversation logic.4. **ElevenLabs API Key**: For realistic text-to-speech.5. **Ngrok** (or similar): To expose your local port 8080 to Twilio.## Setup1. **Install Dependencies**: ```bash pip install -r scripts/requirements.txt ```2. **Set Environment Variables** (in `~/.moltbot/.env`, `~/.clawdbot/.env`, or export): ```bash export DEEPGRAM_API_KEY="your_key" export OPENAI_API_KEY="your_key" export ELEVENLABS_API_KEY="your_key" export TWILIO_ACCOUNT_SID="your_sid" export TWILIO_AUTH_TOKEN="your_token" export PORT=8080 ```3. **Start the Server**: ```bash python3 scripts/server.py ```4. **Expose to Internet**: ```bash ngrok http 8080 ```5. **Configure Twilio**: - Go to your Phone Number settings. - Set "Voice & Fax" -> "A Call Comes In" to **Webhook**. - URL: `https://<your-ngrok-url>.ngrok.io/incoming` - Method: `POST`## UsageCall your Twilio number. The agent should answer, transcribe your speech, think, and reply in a natural voice.## Customization- **System Prompt**: Edit `SYSTEM_PROMPT` in `scripts/server.py` to change the persona.- **Voice**: Change `ELEVENLABS_VOICE_ID` to use different voices.- **Model**: Switch `gpt-4o-mini` to `gpt-4` for smarter (but slower) responses.

Frequently asked questions