Rerun

rerun

The data-centric visualization tool for robium's ML/non-ROS side: a timeline-and-entity-path logging model (Python-first via rerun-sdk, pip install rerun-sdk / uv add rerun-sdk, currently at 0.34.1 as of this session — verified via direct fetch of github.com/rerun-io/rerun's release page) for policy rollouts, LeRobot episode data, and arbitrary sensor streams that don't fit ROS message types. This skill deliberately stays thin: Rerun's own logging API surface (which archetype for which data shape, exact keyword arguments) changes across releases, and its own getting-started docs and example gallery are the actual source of truth — this skill's job is orientation (what the pieces are called, how the operating modes map to robium's local/remote needs) and the hand-off into lerobot for episode visualization, not a re-implementation of Rerun's own tutorials.

When to use this skill

• Logging and viewing ML rollouts, sensor streams, or embeddings/tensors outside ROS's message-type world; visualizing a LeRobot dataset episode.
• The trigger phrases in the description: 'rerun', visualizing ML training/eval rollouts, LeRobot episode data, custom sensor pipelines outside ROS tooling.
• Before writing any rr.log(...) call for a new data shape — check Rerun's own examples first (see Key directives); this skill's job is pointing there, not supplying every archetype from memory.
• Cross-references — go to the sibling skill instead when the question is:
  • LeRobot dataset mechanics themselves (the LeRobotDataset format, loading/recording, training/eval CLI) → lerobot. This skill only covers viewing an episode once lerobot has one to show, not the dataset format itself.
  • Live ROS 2 topic debugging with a local display → rviz2.
  • Live ROS 2 topic debugging remote/headless, or MCAP recording → foxglove.
  • Deciding which dataset to source in the first place → data.
  • Choosing which viz tool fits the situation at all → visualization (routes here once Rerun is the right choice).

Key directives

• Delegation posture: point-upstream (delegate-leaning). Rerun ships its own extensive example gallery and API reference at rerun.io/docs and rerun.io/examples, and the archetype/API surface has moved release to release (see the SDK's own migration guides). Check those before writing logging code for a new data shape — treat this skill as an index into them, not a replacement.
• Never re-teach LeRobot dataset mechanics here. lerobot's lerobot-dataset-viz command already wraps Rerun for episode visualization end to end (see Usage patterns) — this skill's job is receiving that hand-off (what the Rerun-specific pieces mean once the command is running), not duplicating lerobot's dataset/CLI docs.
• Pick the operating mode by where the viewer needs to be, not by habit. Rerun's Python SDK supports several distinct sinks (rr.spawn(), rr.connect_grpc(), rr.save(), rr.serve_grpc()/rr.serve_web_viewer(), rr.stdout()) — local interactive work, a separately-running viewer, a file for later, and a browser-accessible remote view are different modes, not variations of the same one. See Usage patterns and Rerun's own operating modes docs (this session's content came via search-synthesis, not a direct fetch — the URL 404'd; see References — re-verify before relying on the exact operating-mode list) before defaulting to spawn() on a headless box (it will fail — there's no local display to spawn a viewer window on).
• Never write archetype names, constructor arguments, or CLI flags from memory. The archetype list (Points3D, Image, Scalars, Transform3D, and others) and their exact fields change across releases — verify against the rerun.io/docs quick-start/data-in pages or the rerun-io/rerun GitHub repo (fetched directly this session for these facts) before repeating a specific call signature in a real project.

Quick start

1. Install:

uv add rerun-sdk

2. Log something and view it locally — spawn=True starts a Viewer process and streams to it over gRPC:

import rerun as rr

rr.init("my_app", spawn=True)
rr.log("world/points", rr.Points3D(points, colors=colors, radii=0.05))

3. For anything beyond a first smoke test — a new data shape, a timeline, a blueprint/layout — go to Rerun's Python quick-start docs and example gallery before hand-writing more rr.log() calls; see Key directives.

Usage patterns

Log a simple stream. rr.init(app_id, spawn=True) opens a local viewer; each rr.log(entity_path, archetype) call writes one entity at the current time. Entity paths (e.g. "robot/camera/image") form a hierarchy that drives the viewer's tree and default layout — group related data under a common path prefix rather than flat, unrelated names. For anything with a time axis beyond wall-clock logging order, set an explicit timeline first: rr.set_time("episode_step", sequence=i) (or duration=/timestamp= for the other timeline kinds) before the rr.log() calls for that step — Rerun uses "latest-at" semantics, so static data logged once at the start persists across the whole timeline without re-logging it every step. See the quick-start link above for the current archetype list.

Visualize a LeRobot episode. Don't hand-roll this — lerobot already wraps Rerun for exactly this case:

lerobot-dataset-viz --repo-id=<id> --episode-index=0

This is the lerobot skill's command, cross-referenced here because it's the primary way most robium projects will ever invoke Rerun directly. The entity paths and layout it produces are LeRobot's own convention (camera streams, state/action vectors, per-episode structure) — inspect what it logs before adding custom rr.log() calls alongside it, rather than guessing the path scheme.

Remote viewer. Two shapes, matching lerobot-dataset-viz's own --mode distant flag: either start a gRPC server in the logging process (rr.serve_grpc(), optionally paired with rr.serve_web_viewer() for a browser-based viewer with no local install) and connect to it, or start a standalone viewer first (rerun in a terminal) and have the logging process call rr.connect_grpc() to stream into it. From a separate machine, the native viewer connects to a running gRPC server with:

rerun rerun+http://<host>:<grpc_port>/proxy

which is the same pattern lerobot-dataset-viz --mode distant --grpc-port=<port> uses under the hood — a headless training/eval box logs, a local machine watches, with no data ever needing to leave the remote box as a file.

Platform gotchas

• The viewer runs natively everywhere — Linux, macOS (including Apple Silicon), and Windows — unlike rviz2, which needs ROS 2 and therefore Docker on macOS. This is why rerun is the default answer for a non-ROS pipeline regardless of local vs. remote, per the visualization umbrella's selection table.
• spawn() needs a local display; it will fail on a bare headless box. If a training/eval script that calls rr.spawn() is moved to a headless server, switch to rr.serve_grpc()/serve_web_viewer() or rr.save() instead — don't try to get a local window working over SSH.
• The Python SDK bundles the Viewer; C++ and Rust don't. A pure-Python robium project gets the viewer for free via pip install rerun-sdk; a C++/Rust logging path needs the separate rerun-cli (cargo install rerun-cli --locked --features nasm, per the SDK's own install docs) to get the same standalone viewer/CLI.
• serve_grpc() buffers in memory so late-connecting viewers still get full history — on a long-running process logging a lot of data, set its server_memory_limit argument rather than letting it grow unbounded.

Customization

• Entity path scheme for a new project: don't invent one from scratch — Rerun's own docs on entity paths and hierarchies (linked from the quick-start page above) cover the conventions (grouping by sensor/subsystem, using Transform3D logs to place entities in a common coordinate frame); mirror lerobot-dataset-viz's own scheme when logging alongside a LeRobot episode so the two show up coherently in one tree.
• Mixing ROS and non-ROS data in one debugging session: per the visualization umbrella, this is normal — use rviz2/foxglove for the ROS-side state and rerun for a policy's own inputs/outputs, viewed side by side rather than forcing one tool to cover both.

References

• Upstream: Rerun documentation and Python quick-start (primary source for logging code — check before writing new rr.log() calls), operating modes reference (source of the spawn/connect/save/serve/stdout distinction above; the exact URL 404'd on direct fetch this session, so this content came via search-synthesis — re-verify before relying on the exact operating-mode list), rerun-io/rerun GitHub repo (fetched directly this session — source of the current version and rerun-cli install command), example gallery. Sibling skills: lerobot (owns LeRobot dataset mechanics; wraps this skill for episode visualization via lerobot-dataset-viz), data (dataset sourcing strategy), rviz2 and foxglove (ROS-native live debugging, not this skill's territory), visualization (umbrella, routes here).

Changelog