Hub Recording

Data Recording Nodes

Record demonstrations for imitation learning and model fine-tuning

Available Recording Nodes

dora-lerobot Installation

dora-lerobot is from a separate repository, not PyPI:

git clone https://github.com/dora-rs/dora-lerobot
cd dora-lerobot
pip install -e dora_lerobot

llama-factory-recorder

Record data for fine-tuning language and vision-language models with LLaMA Factory.

Prerequisites

git clone https://github.com/hiyouga/LLaMA-Factory --depth 1 $HOME/LLaMA-Factory

YAML Configuration

- id: recorder
  build: pip install llama-factory-recorder
  path: llama-factory-recorder
  inputs:
    image_right: camera/image
    ground_truth: human/text  # Human-provided labels/responses
  outputs:
    - text
  env:
    DEFAULT_QUESTION: "Respond to people."
    LLAMA_FACTORY_ROOT_PATH: $HOME/LLaMA-Factory

Recording Pipeline

nodes:
  # Camera
  - id: camera
    build: pip install opencv-video-capture
    path: opencv-video-capture
    inputs:
      tick: dora/timer/millis/100
    outputs:
      - image

  # Keyboard input for labels
  - id: keyboard
    build: pip install terminal-input
    path: terminal-input
    outputs:
      - text

  # Recorder
  - id: recorder
    build: pip install llama-factory-recorder
    path: llama-factory-recorder
    inputs:
      image_right: camera/image
      ground_truth: keyboard/text
    outputs:
      - text
    env:
      DEFAULT_QUESTION: "What action should the robot take?"
      LLAMA_FACTORY_ROOT_PATH: $HOME/LLaMA-Factory

  # Visualization
  - id: rerun
    build: pip install dora-rerun
    path: dora-rerun
    inputs:
      image: camera/image

Training with Recorded Data

1. Run your dataflow to collect data
2. Data is saved to LLaMA Factory folder
3. Modify training config:

# llama-factory/examples/train_lora/qwen2vl_lora_sft.yaml
dataset: dora_demo_1,identity  # Your recorded dataset
model_name_or_path: Qwen/Qwen2.5-VL-3B-Instruct  # or 7B

4. Train:

llamafactory-cli train examples/train_lora/qwen2vl_lora_sft.yaml

lerobot-dashboard

Pygame-based interface for LeRobot data collection with dual camera display.

YAML Configuration

- id: dashboard
  build: pip install lerobot-dashboard
  path: lerobot-dashboard
  inputs:
    tick: dora/timer/millis/16  # 60fps update
    image_left: camera_left/image
    image_right: camera_right/image
  outputs:
    - text      # User text input
    - episode   # Episode number (-1 marks end)
    - failed    # Failed episode number
    - end       # End signal for dataflow
  env:
    WINDOW_WIDTH: 1280
    WINDOW_HEIGHT: 1080

Outputs

LeRobot Recording Pipeline

nodes:
  # Left camera
  - id: camera_left
    build: pip install opencv-video-capture
    path: opencv-video-capture
    inputs:
      tick: dora/timer/millis/33
    outputs:
      - image
    env:
      PATH: "0"

  # Right camera
  - id: camera_right
    build: pip install opencv-video-capture
    path: opencv-video-capture
    inputs:
      tick: dora/timer/millis/33
    outputs:
      - image
    env:
      PATH: "1"

  # Leader arm (teleoperation)
  - id: leader
    build: pip install dora-piper
    path: dora-piper
    inputs:
      tick: dora/timer/millis/20
    outputs:
      - joint_state
    env:
      MODE: leader

  # Follower arm
  - id: follower
    build: pip install dora-piper
    path: dora-piper
    inputs:
      joint_positions: leader/joint_state
    outputs:
      - joint_state
    env:
      MODE: follower

  # Dashboard
  - id: dashboard
    build: pip install lerobot-dashboard
    path: lerobot-dashboard
    inputs:
      tick: dora/timer/millis/16
      image_left: camera_left/image
      image_right: camera_right/image
    outputs:
      - text
      - episode
      - failed
      - end
    env:
      WINDOW_WIDTH: 1280
      WINDOW_HEIGHT: 720

  # LeRobot recorder (install from dora-lerobot repo first)
  - id: lerobot
    path: dora-lerobot-recorder
    inputs:
      image_left: camera_left/image
      image_right: camera_right/image
      state: follower/joint_state
      action: leader/joint_state
      episode: dashboard/episode
      end: dashboard/end
    env:
      DATASET_NAME: my_robot_dataset

Complete Imitation Learning Pipeline

1. Data Collection

# dataflow_record.yml
nodes:
  - id: camera
    build: pip install opencv-video-capture
    path: opencv-video-capture
    inputs:
      tick: dora/timer/millis/33
    outputs:
      - image

  - id: leader
    build: pip install dora-piper
    path: dora-piper
    inputs:
      tick: dora/timer/millis/20
    outputs:
      - joint_state
    env:
      MODE: leader

  - id: follower
    build: pip install dora-piper
    path: dora-piper
    inputs:
      joint_positions: leader/joint_state
    outputs:
      - joint_state
    env:
      MODE: follower

  - id: dashboard
    build: pip install lerobot-dashboard
    path: lerobot-dashboard
    inputs:
      tick: dora/timer/millis/16
      image_left: camera/image
    outputs:
      - episode
      - end

  # LeRobot recorder (install from dora-lerobot repo first)
  - id: recorder
    path: dora-lerobot-recorder
    inputs:
      image: camera/image
      state: follower/joint_state
      action: leader/joint_state
      episode: dashboard/episode
    env:
      DATASET_NAME: pick_and_place

2. Policy Training (offline)

# Train with LeRobot
python lerobot/train.py \
    --dataset pick_and_place \
    --policy diffusion

3. Policy Deployment

# dataflow_deploy.yml
nodes:
  - id: camera
    build: pip install opencv-video-capture
    path: opencv-video-capture
    inputs:
      tick: dora/timer/millis/33
    outputs:
      - image

  - id: policy
    build: pip install dora-rdt-1b
    path: dora-rdt-1b
    inputs:
      image: camera/image
    outputs:
      - action

  - id: robot
    build: pip install dora-piper
    path: dora-piper
    inputs:
      joint_positions: policy/action
    outputs:
      - joint_state

Data Formats

Episode Markers

# Start new episode
node.send_output("episode", pa.array([episode_number]))

# End episode
node.send_output("episode", pa.array([-1]))

# Mark episode as failed
node.send_output("failed", pa.array([episode_number]))

Recording State/Action Pairs

# State: current robot joint positions
state = pa.array(current_joints)

# Action: commanded joint positions (from leader/policy)
action = pa.array(target_joints)

Related Skills

• hub-robot - Robot control for teleoperation
• hub-camera - Camera input for recording
• data-pipeline - LeRobot data tools