Yolo Ros2 Integration

name: yolo-ros2-integration description: 'Integrate YOLO object detection with ROS 2. Use when running YOLOv8/v5 inference, publishing Detection2D messages, or real-time detection.'

YOLO ROS 2 Integration

Overview

YOLO (You Only Look Once) provides real-time object detection. For ROS 2 Jazzy, the main integration paths are:

1. Direct ultralytics Python API — simplest, run inference in a ROS 2 node
2. yolo_ros2 — community ROS 2 wrapper with lifecycle management
3. OAK-D on-device inference — compile YOLO to a .blob, run on the Myriad X VPU at 15-30 FPS with zero CPU load

Model Selection

For a patrol robot running on a Jetson or RPi with OAK-D, use yolov8n or yolov11n on-device. For a system with a CUDA GPU, yolov8s or yolov8m gives better accuracy.

Output Messages

Standard output: vision_msgs/msg/Detection2DArray

Detection2DArray:
  header: ...
  detections[]:
    - bbox:
        center: {x: 320.0, y: 240.0}
        size_x: 85.0
        size_y: 200.0
      results[]:
        - hypothesis:
            class_id: "person"
            score: 0.92

Minimal Detection Node (ultralytics)

import rclpy
from rclpy.node import Node
from sensor_msgs.msg import Image
from vision_msgs.msg import Detection2DArray, Detection2D, ObjectHypothesisWithPose
from cv_bridge import CvBridge
from ultralytics import YOLO


class YoloDetector(Node):
    def __init__(self):
        super().__init__('yolo_detector')
        self.declare_parameter('model_path', 'yolov8n.pt')
        self.declare_parameter('confidence_threshold', 0.5)
        self.declare_parameter('device', 'cpu')  # 'cpu', 'cuda:0', 'mps'

        model_path = self.get_parameter('model_path').value
        self.conf_thresh = self.get_parameter('confidence_threshold').value
        device = self.get_parameter('device').value

        self.model = YOLO(model_path)
        self.model.to(device)
        self.bridge = CvBridge()

        self.sub = self.create_subscription(Image, '/camera/image_raw', self.image_cb, 10)
        self.pub = self.create_publisher(Detection2DArray, '/detections', 10)

    def image_cb(self, msg):
        cv_image = self.bridge.imgmsg_to_cv2(msg, desired_encoding='bgr8')
        results = self.model(cv_image, conf=self.conf_thresh, verbose=False)

        det_array = Detection2DArray()
        det_array.header = msg.header

        for result in results:
            for box in result.boxes:
                det = Detection2D()
                x1, y1, x2, y2 = box.xyxy[0].tolist()
                det.bbox.center.position.x = (x1 + x2) / 2.0
                det.bbox.center.position.y = (y1 + y2) / 2.0
                det.bbox.size_x = x2 - x1
                det.bbox.size_y = y2 - y1

                hyp = ObjectHypothesisWithPose()
                hyp.hypothesis.class_id = result.names[int(box.cls[0])]
                hyp.hypothesis.score = float(box.conf[0])
                det.results.append(hyp)
                det_array.detections.append(det)

        self.pub.publish(det_array)


def main():
    rclpy.init()
    rclpy.spin(YoloDetector())

Inference Backends

CPU: Default. Suitable for yolov8n at ~10 FPS on modern x86. On ARM (RPi 5), expect 2-3 FPS.

CUDA: Install ultralytics with CUDA support. Set device='cuda:0'. Achieves 30+ FPS on Jetson Orin with yolov8s.

OAK-D (Myriad X): Compile the model to OpenVINO IR then to .blob:

# Export to ONNX
yolo export model=yolov8n.pt format=onnx imgsz=416

# Convert to blob (use blobconverter or OpenVINO toolkit)
pip install blobconverter
python -c "
import blobconverter
blob_path = blobconverter.from_onnx(
    model='yolov8n.onnx',
    data_type='FP16',
    shaves=6,
    output_dir='.',
)
print(f'Blob: {blob_path}')
"

On-device inference runs the entire detection pipeline on the OAK-D's VPU. The host receives detection results only — no image transfer needed for detection. Achieves 15-30 FPS with zero CPU utilization.

NMS and Confidence Tuning

• Confidence threshold (conf): Minimum score to keep a detection. Default 0.25; raise to 0.5+ for patrol use to reduce false positives.
• NMS IoU threshold (iou): When overlapping boxes exist, suppress those with IoU above this threshold. Default 0.7; lower to 0.45 to be more aggressive.

results = self.model(frame, conf=0.5, iou=0.45, classes=[0])  # class 0 = person

ONNX Export for Cross-Platform

yolo export model=yolov8n.pt format=onnx imgsz=640 simplify=True

ONNX models run via onnxruntime on CPU, CUDA, TensorRT, or OpenVINO. This decouples the model from the PyTorch dependency:

import onnxruntime as ort
session = ort.InferenceSession('yolov8n.onnx', providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])

Launch Configuration

Node(
    package='my_perception',
    executable='yolo_detector',
    name='yolo_detector',
    parameters=[{
        'model_path': '/opt/models/yolov8n.pt',
        'confidence_threshold': 0.5,
        'device': 'cpu',
    }],
    remappings=[
        ('/camera/image_raw', '/oakd/rgb/image'),
    ],
),