Agent Ros Bridge

Agent ROS Bridge - Intelligent Robot Control with Real-Time Feedback

Control robots through natural language conversations with Agent ROS Bridge's universal gateway.

Quick Start

Start the Bridge

# Generate a secure secret
export JWT_SECRET=$(openssl rand -base64 32)

# Start the bridge
agent-ros-bridge --websocket-port 8765

Connect and Control

Option 1: Natural Language (Recommended)

"Move forward 2 meters"
"Turn left 90 degrees"
"Navigate to the kitchen"
"What do you see?"

Option 2: Direct API

import asyncio
import websockets
import json

async def control_robot():
    uri = "ws://localhost:8765?token=<JWT_TOKEN>"
    async with websockets.connect(uri) as ws:
        await ws.send(json.dumps({
            "command": {"action": "move", "parameters": {"direction": "forward", "distance": 2.0}}
        }))
        response = await ws.recv()
        print(json.loads(response))

asyncio.run(control_robot())

Natural Language Commands

Movement Commands

Basic Movement

Intelligent Navigation

Parameter Inference:

• Speed: "slowly" = 0.1 m/s, "normal" = 0.5 m/s, "fast" = 1.0 m/s
• Distance: "a bit" = 0.5m, "a little" = 0.3m, specific numbers used exactly
• Direction: Context-aware (forward = current heading)

Sensor Queries

Visual Perception

Status Queries

Fleet Management

Robot Selection

Multi-Robot Coordination

Safety Commands

Emergency Actions

Safety Queries

Common Tasks by Scenario

Home Environment

Morning Routine:

"Check all rooms"
"Bring me coffee from the kitchen"
"Check if doors are locked"
"Water the plants"

Security Patrol:

"Patrol the house every hour"
"Check for unusual sounds"
"Monitor the front door"
"Alert me if you see anyone"

Office Environment

Delivery Tasks:

"Deliver this package to Room 302"
"Collect documents from the printer"
"Bring coffee to the conference room"
"Distribute mail to all offices"

Meeting Support:

"Escort the visitor to the meeting room"
"Prepare the conference room"
"Check AV equipment status"
"Guide guests to their seats"

Warehouse/Industrial

Inventory:

"Scan shelf A-12"
"Count items in zone B"
"Find item SKU-12345"
"Report low stock items"

Logistics:

"Transport pallet to shipping dock"
"Organize incoming inventory"
"Prepare order for shipment"
"Load truck bay 3"

Advanced Features

Context Awareness

The skill maintains context across conversations:

User: "Go to the kitchen"
[Robot navigates to kitchen]

User: "Now bring me water"
[Robot knows it's in kitchen, searches for water]

User: "Return to base"
[Robot knows to return to starting position]

Learning & Memory

Location Learning:

User: "Remember this as 'charging station'"
[Robot saves current location]

User: "Go to charging station"
[Robot navigates to learned location]

Path Learning:

User: "Learn this route to the office"
[Robot records path]

User: "Take the learned route"
[Robot follows recorded path]

Autonomous Behaviors

Exploration:

"Explore autonomously for 10 minutes"
"Map this area"
"Find all rooms"
"Identify objects of interest"

Patrol:

"Patrol every 30 minutes"
"Check for anomalies"
"Report anything unusual"
"Maintain security perimeter"

Safety & Best Practices

Dangerous Actions (Require Confirmation)

These actions trigger safety confirmation:

• Movement in confined spaces (< 1m clearance)
• Speed > 1.5 m/s
• Operating near humans without detection
• Multi-robot coordination in tight spaces
• Autonomous exploration in unknown areas

Emergency Procedures

Immediate Stop:

• Say "Emergency stop" or "Stop now"
• No confirmation required
• Halts all motion immediately

Recovery:

• "Resume previous task" - Continue after stop
• "Return to safe position" - Go to known safe location
• "Reset and start over" - Clear state and restart

Safety Checklist

Before autonomous operations:

1. ✓ Environment scanned for obstacles
2. ✓ Humans accounted for (if present)
3. ✓ Emergency stop accessible
4. ✓ Battery sufficient for task
5. ✓ Communication link verified

Technical Reference

Connection Details

WebSocket:

• URL: ws://localhost:8765?token=<JWT>
• Protocol: JSON messages
• Auth: JWT token required

MQTT:

• Broker: Configurable (default: localhost:1883)
• Topics: agent_ros_bridge/commands, agent_ros_bridge/telemetry

gRPC:

• Port: 50051 (configurable)
• TLS: Supported with certificates

Message Format

Command:

{
  "command": {
    "action": "move",
    "parameters": {
      "direction": "forward",
      "distance": 1.0,
      "speed": 0.5
    }
  }
}

Response:

{
  "status": "success",
  "data": {
    "action": "move",
    "completed": true,
    "duration": 2.5
  }
}

Available Actions

Movement:

• move - Linear movement
• rotate - Angular rotation
• navigate - Goal-based navigation
• follow - Person following
• explore - Autonomous exploration

Sensors:

• get_camera_image - Capture camera frame
• get_lidar_scan - Get LiDAR data
• get_odometry - Position and velocity
• detect_objects - Object detection

Fleet:

• list_robots - Available robots
• select_robot - Choose best robot
• coordinate - Multi-robot task
• get_fleet_status - Overall status

Safety:

• emergency_stop - Immediate halt
• get_safety_status - Safety system state
• confirm_safe - Safety verification

Troubleshooting

Common Issues

"Connection refused"

• Check if bridge is running
• Verify port 8765 is available
• Check firewall settings

"Authentication failed"

• Verify JWT token is valid
• Check token hasn't expired
• Ensure JWT_SECRET matches

"Robot not responding"

• Check ROS connection
• Verify robot is powered on
• Check for error messages

"Navigation failed"

• Check if map is loaded
• Verify goal is reachable
• Check for obstacles

Debug Commands

# Check bridge status
"Get bridge status"

# Check robot connection
"Verify robot connection"

# Test sensors
"Test all sensors"

# Diagnostic report
"Generate diagnostic report"

Real-Time OpenClaw Integration

Agent ROS Bridge provides bidirectional real-time communication with OpenClaw and Web UI through the OpenClaw Bridge.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                     Real-Time Data Flow                          │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│   OpenClaw Agent        OpenClaw Bridge      Agent ROS Bridge   │
│        │                       │                       │         │
│        │  Command             │                       │         │
│        │ ───────────────────> │                       │         │
│        │                       │  Forward Command      │         │
│        │                       │ ───────────────────> │         │
│        │                       │                       │         │
│        │  Telemetry           │  Robot Telemetry      │         │
│        │ <─────────────────── │ <─────────────────── │         │
│        │                       │                       │         │
│        │                       │                       │         │
│   Web UI Clients              │                       │         │
│        │                       │                       │         │
│        │  Status/Control      │                       │         │
│        │ <──────────────────> │                       │         │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Starting the OpenClaw Bridge

# Start Agent ROS Bridge first
agent-ros-bridge --websocket-port 8765

# Start OpenClaw Bridge
python skills/agent-ros-bridge/scripts/openclaw_bridge.py --port 8766

OpenClaw Real-Time Commands

Live Telemetry Streaming

# OpenClaw receives continuous telemetry updates
"Start monitoring robot telemetry"
→ Streaming: battery, position, velocity, sensor data
→ Updates every 500ms (2Hz)

Command with Progress Updates

# Commands send real-time progress to OpenClaw
"Navigate to the kitchen"
→ "Navigating... 10% complete"
→ "Navigating... 45% complete - avoiding obstacle"
→ "Navigating... 90% complete"
→ "Arrived at kitchen"

Multi-Client Synchronization

# Web UI and OpenClaw stay in sync
"All clients see the same robot state"
→ OpenClaw sees: "Robot moving to kitchen"
→ Web UI shows: Live map with robot position
→ Both update simultaneously

WebSocket Events

From Robot to OpenClaw/Web UI

{
  "type": "telemetry_update",
  "data": {
    "position": {"x": 1.5, "y": 2.0, "theta": 0.5},
    "battery": 85,
    "velocity": {"linear": 0.5, "angular": 0.0},
    "sensors": {"lidar": "active", "camera": "active"}
  },
  "timestamp": "2026-04-06T14:20:00Z"
}

Command Result

{
  "type": "command_result",
  "command": {"action": "navigate", "target": "kitchen"},
  "response": {
    "status": "success",
    "execution_time": 12.5,
    "path_deviation": 0.1
  },
  "timestamp": "2026-04-06T14:20:12Z"
}

Emergency Events

{
  "type": "emergency_stop",
  "source": "web_ui",
  "reason": "user_triggered",
  "timestamp": "2026-04-06T14:20:30Z"
}

OpenClaw Skill Usage

from skills.agent_ros_bridge.scripts.openclaw_bridge import OpenClawSkillInterface

async def control_robot():
    # Connect to bridge
    interface = OpenClawSkillInterface(port=8766)
    await interface.connect()
    
    # Send command with real-time feedback
    result = await interface.send_command(
        "Navigate to charging station",
        speed="normal"
    )
    
    # Get live telemetry
    telemetry = await interface.get_telemetry()
    print(f"Battery: {telemetry['battery']}%")
    print(f"Position: {telemetry['position']}")
    
    # Query status
    status = await interface.query_status()
    
    await interface.disconnect()

Web UI Integration

The Web UI connects to the same bridge for synchronized control:

// Web UI connection
const ws = new WebSocket('ws://localhost:8766', ['openclaw-bridge']);

ws.onopen = () => {
  ws.send(JSON.stringify({ client_type: 'web_ui' }));
};

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);
  
  if (data.type === 'telemetry_update') {
    updateMap(data.data.position);
    updateBatteryIndicator(data.data.battery);
  }
  
  if (data.type === 'command_result') {
    showNotification('Command completed');
  }
};

Examples Gallery

Example 1: Morning House Check

User: "Check the house"
→ Robot: "Starting house check. I'll visit each room and report."

[Robot moves through rooms]

Robot: "House check complete. Kitchen: normal. Living room: window 
        left open. Bedroom: normal. No issues detected."

Example 2: Office Delivery

User: "Deliver this to Alice in Room 302"
→ Robot: "I'll deliver to Alice in Room 302. Navigating..."

[Robot navigates, finds Alice]

Robot: "Delivery complete. Alice acknowledged receipt. Returning to base."

Example 3: Fleet Coordination

User: "Search the warehouse for item SKU-12345"
→ Robot: "Coordinating search with 3 robots. Dividing area..."

[Robots search in parallel]

Robot: "Item found by Robot-2 in Zone C, Shelf 12. All robots returning."

References

• ROS1 Guide: See references/ros1-guide.md for ROS1-specific details
• ROS2 Guide: See references/ros2-guide.md for ROS2-specific details
• API Reference: See docs/API_REFERENCE.md for complete API
• Architecture: See docs/ARCHITECTURE_V2.md for system design

Version Info

• Skill Version: 0.6.7
• Agent ROS Bridge: 0.6.7
• OpenClaw Bridge: Real-time bidirectional communication
• ROS1 Support: Noetic
• ROS2 Support: Jazzy, Humble
• Last Updated: 2026-04-10

Quick Reference