Dear ImGui Development & Architecture Guide
[!TIP]
Official MuJoCo documentation (doc/*.rst) & related skills:
../../studio/python_plugins.rst
(.. _StudioUiPlugins:, .. _StudioImGuiTips:) and
../../studio/architecture.rst
(.. _StudioCppComponents:) — How Dear ImGui and ImPlot integrate into
MuJoCo Studio, UpdateEvent vs BuildGuiEvent, and C++ (ux/) vs Python
widget trade-offs. Read these short .rst chapters directly when present
locally or in pip install mujoco under
pathlib.Path(mujoco.__file__).parent / 'doc', or fetch
https://mujoco.readthedocs.io/en/latest/<path>.html (or /en/stable/
once released; Sphinx anchors lowercase the label, e.g.
#studioimguitips).
../../programming/ui.rst (.. _UI:) —
Legacy mjui C framework used only in classic simulate.cc (distinct
from Studio's Dear ImGui stack).
- mujoco-studio — Studio viewer launchers, decoupled
sim/viewer threads, plugins, hot reload, and
WebViewer streaming.
[!NOTE]
Dear ImGui is an immediate-mode GUI framework designed for fast iterations,
minimal state overhead, and high responsiveness. This guide documents best
practices and conventions for writing maintainable, performant, and
testable UI code across MuJoCo, its Python bindings, and dependent targets
in C++, Python, and WebViewer.
Applies to both C++ and Python: All architectural guidelines, widget
patterns, Tables API usage, ID stack rules (## vs ###), and performance
principles apply equally whether authoring native C++ (ImGui::) or Python
(imgui.) code. While snippets frequently illustrate patterns in Python for
concise readability, the corresponding immediate-mode concepts, widget APIs,
ID stack mechanics, clipper usage, and styling rules translate directly 1-to-1
to C++. Guidelines apply both to core Dear ImGui and extension libraries, most
importantly ImPlot.
1. Naming Conventions: build_gui vs render / draw
ImGui code does not render pixels to the screen directly. Instead, calling ImGui
functions builds and records draw command lists (vertex buffers, index
buffers, clip rects, and texture references) that are processed, rasterized,
and rendered later in the frame pipeline (via OpenGL, Vulkan, or
WebGL/WebViewer).
- Use
build_gui() or build_xxx_gui() (in Python) or BuildGui() /
BuildXxxGui() (in C++) for all functions and methods that submit ImGui
widgets (e.g., build_control_panel_gui(), BuildControlPanelGui()).
- Avoid
render() or draw() for UI layout functions. Reserve render /
draw strictly for low-level graphics backend operations (such as
ImGui_ImplOpenGL3_RenderDrawData).
# Good: Accurately describes building the frame's UI hierarchy
def build_gui(self) -> None:
self.build_header_gui()
self.build_table_gui()
# Avoid: Suggests immediate rasterization/rendering
def draw_ui(self):
...
2. Decoupling GUI from Application Logic
GUI code must strictly handle user layout, presentation, and input forwarding.
The GUI is intended solely to glue the underlying logic together, and that glue
layer should be as small and thin as possible. Never mix business logic,
network/RPC queries, file operations, or simulation steps directly into frame
build functions.
- Minimal Glue Layer: The GUI component should do little more than bind
user inputs to underlying library calls and render current state.
- Separate Libraries: Core business logic belongs in a standalone library
or module (e.g.,
inspector_lib.py vs inspector_gui.py).
- Asynchronous Execution: Expensive operations (RPCs, subprocesses, large
searches) must run on worker threads/tasks, with GUI code reading cached
state or progress flags.
- Testability: Decoupled libraries can be thoroughly tested via standard
unit tests without requiring an active ImGui context or window.
3. Custom Widgets: Immediate Mode Pure Functions
Custom widgets should remain lightweight, immediate-mode pure functions:
- Pure Immediate Mode: Pass inputs as arguments, return output or mutation
tuples (e.g.,
changed, new_val = dial_widget(...)). Avoid heavy stateful
framework abstractions.
- Avoid Per-Widget Persistent State: Storing state across frames inside a
widget incurs a maintenance burden and limits reusability across arbitrary
contexts. Keep widgets stateless whenever possible; if state is essential,
store it in the parent application state rather than inside the widget.
- UX Customizations & Libraries: Tailored visual controls (e.g., dial
knobs, meters) are encouraged when they enhance UX. Instead of importing
large external dependencies, copy and refactor small, focused snippets
(must be MIT-licensed) into clean immediate-mode functions. In MuJoCo,
shared custom ImGui widget functionality lives in the
imgui_widgets.h/cc files in the Studio UX module.
- Test Engine Readiness: Expose interaction endpoints and bounding boxes
using Dear ImGui Test Engine hooks/macros so automated test drivers can
interact with custom controls.
- Check Upstream & Changelog Before Implementing: Before writing a new
custom widget, layout behavior, or feature from scratch:
- Check the latest upstream changelog for the
docking branch at
https://github.com/ocornut/imgui/blob/docking/docs/CHANGELOG.txt and
inspect what has changed relative to the vendored version in MuJoCo.
- Search Dear ImGui GitHub issues
(https://github.com/ocornut/imgui/issues) and the Wiki
(https://github.com/ocornut/imgui/wiki) for existing solutions, standard
snippets, or discussions on the topic.
- Check Local vs Upstream (Update Dependency): In general, check both
the local checked-out code and the latest code on GitHub. If the
feature, widget, or bugfix already exists in a newer upstream release,
the cleaner solution is often to update the vendored dependency rather
than maintaining custom workarounds in application code.
4. Disallowing Obsolete Functions & Internal Headers
ImGui maintains a stable public API and clearly documents deprecations:
- Ban Obsolete Functions: Always define
IMGUI_DISABLE_OBSOLETE_FUNCTIONS
(and IMGUI_DISABLE_OBSOLETE_KEYIO) during builds and framework updates to
prevent deprecated patterns from lingering in the codebase.
- Public API Stability: Rely on the official public API (
imgui.h). Avoid
including internal headers (imgui_internal.h) in application and tool code
unless implementing foundational backend extensions.
5. Performance & Virtualization: ImGuiListClipper
When rendering lists, tables, or log views that may contain many items (e.g.,
100+ bodies, simulation trajectories, log buffers):
- Always Virtualize with
ImGuiListClipper / imgui.ListClipper:
Without clipping, ImGui generates vertices and text layouts for every item
in the list even when offscreen. In remote streaming backends
(WebViewer), unclipped tables saturate the WebSocket stream. In C++, use
ImGuiListClipper clipper; clipper.Begin(...); in Python, use
clipper = imgui.ListClipper(); clipper.Begin(...).
- Clipper Usage: The clipper calculates the visible range and only submits
the items currently in view.
if imgui.BeginTable('ItemsTable', 1):
clipper = imgui.ListClipper()
clipper.Begin(len(items))
while clipper.Step():
for i in range(clipper.DisplayStart, clipper.DisplayEnd):
imgui.TableNextRow()
imgui.TableNextColumn()
imgui.TextUnformatted(items[i].name)
imgui.EndTable()
- Memory & Lifetime (
imgui.TextUnformatted): Use imgui.TextUnformatted
when displaying large or read-only text strings to avoid string copy and
formatting overhead in Python bindings.
6. Runtime IDs, the ID Stack, and Test Automation (## vs ###)
Dear ImGui identifies widgets by hashing their label text against the current ID
stack seed (ImGuiWindow::GetID calling ImHashStr in imgui.cpp).
Understanding the string operators ## and ### is critical for avoiding ID
collisions, maintaining state across dynamic label changes, and writing
automatable UIs.
Technical Mechanics: ## vs ###
Both operators rely on Dear ImGui's internal text-rendering and hashing
routines:
- Display Boundary (
FindRenderedTextEnd): Text rendering stops at the
first occurrence of ##. Because ### also begins with ##, any
characters following either ## or ### are hidden from the user.
- Hidden Disambiguator (
##):
ImHashStr hashes the entire string ("Label##Suffix"), including
both the visible label and the hidden suffix.
- Purpose: Disambiguating widgets that share the same visible label
within the same ID scope (e.g., multiple "Delete" buttons or repeating
row entries).
- Why
Selectable(f'{body_name}##body_row_{i}') uses ##: The
visible label is body_name, but lists may contain duplicate body
names or repeating entries. Appending ##body_row_{i} hashes the full
string "{body_name}##body_row_{i}", producing a unique ID while
keeping only body_name visible on screen. (Alternatively, wrap row
rendering in imgui.PushID(i) / imgui.PopID()). If ### were used
here instead, body_name would be discarded completely from the hash.
- ID Override / Seed Reset (
###):
ImHashStr specifically detects three # characters (###). When
encountered, it resets the hash seed back to the initial seed
(crc = seed) and hashes only the substring following ###. Any
preceding visible text is completely discarded from the hash.
- Dynamic Labels: When visible text changes across frames (e.g.,
f'Items ({count})###ItemCounterBtn' or animated frame titles), ###
guarantees that the widget's internal ID remains constant. This
preserves active focus, hover state, window docking positions, and
collapse states.
- Semantically Meaningful Names for Automation: For icon-only buttons
(e.g.,
f'{ICON_FA_PLAY}###PlayButton', where
ICON_FA_PLAY = '\uf04b'), the icon glyph itself is constant across
frames and would technically produce a stable hash even without ###.
However, without ###PlayButton, the resulting ID is derived from an
opaque unicode codepoint. Using ###PlayButton assigns a
human-readable, semantically meaningful identifier that facilitates
agent-driven Dear ImGui Test Engine UI automation and computer control
to reliably locate, inspect, and click the control by name
("PlayButton").
[!NOTE]
Dear ImGui Test Engine is not currently bundled with MuJoCo, but support is
planned, along with recorded UI interaction testing.
# Icon glyph constant (e.g. FontAwesome play button unicode "\uf04b")
ICON_FA_PLAY = '\uf04b'
# Semantic ID override: The glyph is displayed, but the ID hashes only
# 'PlayButton'. Provides a meaningful identifier for test automation agents and
# tooling.
imgui.Button(f'{ICON_FA_PLAY}###PlayButton')
# Dynamic label with stable ID: Visible text changes with 'count', but the ID
# remains hashed from 'ItemCounterBtn', preserving focus, click, and hover
# state.
imgui.Button(f'Items ({count})###ItemCounterBtn')
# Disambiguated duplicate label: Visible text is 'body_name', but the ID hashes
# the full string '{body_name}##body_row_{i}' to prevent ID collisions across
# rows.
imgui.Selectable(f'{body_name}##body_row_{i}', is_selected)
# ID Scoping alternative: Using PushID / PopID for row loops instead of ##
# suffixes
imgui.PushID(i)
imgui.Selectable(body_name, is_selected)
imgui.PopID()
Further ImGui & ImPlot References
Consult the upstream Dear ImGui and ImPlot sources and guides for authoritative
details and patterns:
- Dear ImGui ID Stack & API Guide:
imgui.h (around line 600) covers ID
stack operations (PushID, PopID, GetID) and widget disambiguation
rules.
- Dear ImGui Programmer Guide & String Hashing: The top of
imgui.cpp
contains the comprehensive Programmer Guide; lines ~2470–2515 contain
ImHashStr documenting ### seed resetting.
- Dear ImGui FAQ: The Dear ImGui FAQ
includes dedicated sections on "How can I have multiple widgets with the
same label?" and "Why is my widget not reacting when I click on it?".
- Interactive ImGui Demo Window:
imgui_demo.cpp provides complete,
working examples of every widget, table feature, and clipper pattern. Call
imgui.ShowDemoWindow() in your application to explore features
interactively.
- ImPlot Header & API Guide:
implot.h documents all plotting APIs,
flags, styling properties, and context management.
- Interactive ImPlot Demo Window:
implot_demo.cpp contains extensive
interactive demos for lines, scatter, bars, shaded plots, and realtime
scrolling subplots. Call implot.ShowDemoWindow() to inspect
implementations.
7. UI Design & Aesthetics: Grid Alignment (Tables API), Tooltips & Theming Costs
High-quality ImGui tools balance immediate-mode simplicity with clean visual
structure and consistent styling:
Grid-Aligned Widgets & Form Rows (Prefer the Tables API):
- When arranging widgets into grids, key-value inspector panels, parameter
sheets, or multi-row control bars where labels, dropdowns, or sliders
must line up vertically and share equal widths, prefer the modern
Tables API (
imgui.BeginTable, TableNextRow, TableNextColumn).
- Avoid legacy or brittle alignment techniques such as
imgui.Columns()
(obsolete), chaining SameLine() after variable-length text labels
(different label widths shift every subsequent control horizontally),
hardcoding pixel widths with SetNextItemWidth(px), or inserting dummy
spacing.
- Form & Multi-Control Alignment Recipe:
- Pass
int(imgui.TableFlags.SizingStretchProp) | int(imgui.TableFlags.NoPadOuterX)
for borderless layout tables so the outer left and right edges stay
flush with surrounding non-table widgets.
- Place text labels in
imgui.TableColumnFlags.WidthFixed columns
(omitting an explicit width so ImGui auto-fits to the widest label
across all rows) and call imgui.AlignTextToFramePadding() before
TextUnformatted(...) so label baselines center vertically against
framed controls (Combo, Slider, Input).
- Place controls (
Combo, SliderInt, SliderFloat, InputFloat)
in imgui.TableColumnFlags.WidthStretch columns using hidden ##
labels and imgui.SetNextItemWidth(-1) so every control in the
column starts at the exact same X coordinate and stretches to the
exact same length. If sliders also have trailing right-hand labels
of different lengths (e.g., 'Quality' vs 'Overlay'), place those
trailing labels in their own WidthFixed column—calling
SetNextItemWidth(-1) directly on a widget with a visible label
sizes the frame box to the full column width and clips the label.
Tooltip Length & Aspect Ratio:
- Keep tooltips concise—avoid overly wordy descriptions that stretch into
a wide single-line banner across the viewport.
- Because
SetTooltip and SetItemTooltip do not wrap text
automatically, insert explicit newlines (\n) to keep the tooltip box
at a pleasing, compact aspect ratio (~25–40 characters per line across
2–3 lines) rather than a long horizontal strip.
- Split lines at natural phrase, clause, or sentence boundaries (for
example, between the primary action summary and secondary status or
context) so each line reads cleanly on its own.
Costs of Overriding Colors & Styles (Theming Impact):
- Dear ImGui manages color palettes and visual metrics globally
(supporting dark mode, light mode, or custom studio themes). Semantic
colors are queried via
ImGui::GetStyleColorVec4(...) in C++ or
imgui.GetStyleColorVec4(...) in Python.
- When custom widgets hardcode local color or style overrides (via
imgui.PushStyleColor or imgui.PushStyleVar), those values become
decoupled from the global theme defaults.
- If an application or viewer switches themes (such as toggling between
dark and light modes) or updates global styling metrics, hardcoded local
overrides will not adapt and cannot be edited or updated via global
defaults, often causing unreadable contrast or visual inconsistencies.
- Guideline: Rely on semantic ImGui color tokens (
imgui.Col.Text,
imgui.Col.FrameBg, imgui.Col.Button, imgui.Col.Header, etc.) or
derive dynamic colors via imgui.GetStyleColorVec4(...) rather than
hardcoding static hex/RGB values. If a local override is strictly
necessary, always pop it cleanly (PopStyleColor / PopStyleVar) and
ensure it renders legibly across all supported themes.
# Good: Multi-row control bar with vertically aligned labels and equal-length
# dropdowns/sliders using a borderless layout table
flags = int(imgui.TableFlags.SizingStretchProp) | int(imgui.TableFlags.NoPadOuterX)
if imgui.BeginTable('##controls_layout', 4, flags):
imgui.TableSetupColumn('Label', int(imgui.TableColumnFlags.WidthFixed))
imgui.TableSetupColumn('Combo', int(imgui.TableColumnFlags.WidthStretch), 1.1)
imgui.TableSetupColumn('Slider', int(imgui.TableColumnFlags.WidthStretch), 0.9)
imgui.TableSetupColumn('TrailingLabel', int(imgui.TableColumnFlags.WidthFixed))
imgui.TableNextRow()
imgui.TableNextColumn()
imgui.AlignTextToFramePadding()
imgui.TextUnformatted('Format:')
imgui.TableNextColumn()
imgui.SetNextItemWidth(-1)
_, fmt_idx = imgui.Combo('##format', fmt_idx, ['Lossless', 'Lossy'])
imgui.TableNextColumn()
imgui.SetNextItemWidth(-1)
_, quality = imgui.SliderInt('##quality', quality, 1, 100)
imgui.TableNextColumn()
imgui.AlignTextToFramePadding()
imgui.TextUnformatted('Quality')
imgui.TableNextRow()
imgui.TableNextColumn()
imgui.AlignTextToFramePadding()
imgui.TextUnformatted('Viewport:')
imgui.TableNextColumn()
imgui.SetNextItemWidth(-1)
_, cam_idx = imgui.Combo('##camera', cam_idx, camera_names)
imgui.TableNextColumn()
imgui.SetNextItemWidth(-1)
_, opacity = imgui.SliderFloat('##overlay', opacity, 0.0, 1.0)
imgui.TableNextColumn()
imgui.AlignTextToFramePadding()
imgui.TextUnformatted('Overlay')
imgui.EndTable()
# Avoid: Hardcoding colors detaches widgets from global themes
# (breaks dark/light modes)
# imgui.PushStyleColor(imgui.Col.Text, (0.1, 0.1, 0.1, 1.0)) # Unreadable in dark mode!
# Good: Rely on semantic theme colors from ImGui
text_color = imgui.GetStyleColorVec4(imgui.Col.Text)
8. Searchability of UI Strings in Source Code
Developers and agents often search the codebase for text seen on screen to
locate the corresponding feature implementation:
- Single-Line Static Strings: Keep user-visible label strings, button
names, and headings on a single source line rather than breaking phrases
across lines.
- Conservative Interpolation: Avoid over-fragmenting strings or splitting
prefixes/suffixes across multiple format variables when a single cohesive
string or format template makes the UI text searchable.
9. Python Bindings
- Thin Binding Layer (No GUI/Application Logic in Bindings): Just as GUI
code should avoid containing heavy application logic in glue layers,
pybind11 bindings (e.g., ux.cc) must remain thin wrappers around C++
functions (such as ux/*.h and ux/*.cc). Do not write multi-widget ImGui
layout or application logic directly inside Python binding lambdas;
implement the widget or helper in C++ and bind the function instead.
- Add On-Demand (Never Remove): Expose new ImGui or ImPlot bindings to
Python only when they are actually needed and used by an application or
plugin. Once added, do not remove them.
- Order Parity: When exposing new functions, add them in the exact order
they appear in the upstream C++ headers to simplify future version upgrades
and diffs.
- Keyboard Capture Guarding (
io.WantCaptureKeyboard): When implementing
keyboard shortcuts (such as Space for pause or Backspace for reset), always
check not imgui.GetIO().WantCaptureKeyboard before handling the key event.
When a text input, filter box, or popup is active, ImGui sets
WantCaptureKeyboard = True; ignoring this flag causes keystrokes typed
into an input field to trigger simulation actions.
- Initial Focus in Search & Popups (
SetKeyboardFocusHere): When opening
a command palette, filter box, or modal dialog, call
imgui.SetKeyboardFocusHere() immediately before the imgui.InputText or
InputTextWithHint call so the input field receives focus immediately
without requiring a manual click.
1---2name: mujoco-gui3description: >-4 Dear ImGui and ImPlot user interface design for MuJoCo Studio and MuJoCo5 viewer plugins (C++ ux/ components and Python ViewerPlugin build_gui panels):6 immediate-mode widget layout, multi-row widget/column alignment (Tables API),7 tooltip length and aspect ratio, build_gui vs render separation, ID stack8 rules (## label disambiguation vs ### seed reset override), list9 virtualization (ImGuiListClipper), theming, and input capture10 (WantCaptureKeyboard). Use only when building, aligning, styling, or11 debugging Dear ImGui widgets and ImPlot charts inside MuJoCo Studio, MuJoCo12 viewer plugins, or MuJoCo/robotics simulation and control tools. Don't use13 for general-purpose C++ or Python Dear ImGui / ImPlot / imgui_bundle desktop14 applications unrelated to MuJoCo or robotics, or for Studio plugin15 lifecycle/messaging (use mujoco-studio) or core physics stepping (use16 mujoco-python).17metadata:18 source: https://skillmd.com/skills/google-deepmind/mujoco-gui19---2021# Dear ImGui Development & Architecture Guide2223> [!TIP]24>25> **Official MuJoCo documentation (`doc/*.rst`) & related skills:**26>27> - [`../../studio/python_plugins.rst`](../../studio/python_plugins.rst)28> (`.. _StudioUiPlugins:`, `.. _StudioImGuiTips:`) and29> [`../../studio/architecture.rst`](../../studio/architecture.rst)30> (`.. _StudioCppComponents:`) — How Dear ImGui and ImPlot integrate into31> MuJoCo Studio, `UpdateEvent` vs `BuildGuiEvent`, and C++ (`ux/`) vs Python32> widget trade-offs. Read these short `.rst` chapters directly when present33> locally or in `pip install mujoco` under34> `pathlib.Path(mujoco.__file__).parent / 'doc'`, or fetch35> `https://mujoco.readthedocs.io/en/latest/<path>.html` (or `/en/stable/`36> once released; Sphinx anchors lowercase the label, e.g.37> `#studioimguitips`).38> - [`../../programming/ui.rst`](../../programming/ui.rst) (`.. _UI:`) —39> Legacy `mjui` C framework used only in classic `simulate.cc` (distinct40> from Studio's Dear ImGui stack).41> - [mujoco-studio](../studio/SKILL.md) — Studio viewer launchers, decoupled42> sim/viewer threads, plugins, hot reload, and `WebViewer` streaming.4344> [!NOTE]45>46> Dear ImGui is an immediate-mode GUI framework designed for fast iterations,47> minimal state overhead, and high responsiveness. This guide documents best48> practices and conventions for writing maintainable, performant, and49> testable UI code across MuJoCo, its Python bindings, and dependent targets50> in C++, Python, and WebViewer.51>52> **Applies to both C++ and Python**: All architectural guidelines, widget53> patterns, Tables API usage, ID stack rules (`##` vs `###`), and performance54> principles apply equally whether authoring native C++ (`ImGui::`) or Python55> (`imgui.`) code. While snippets frequently illustrate patterns in Python for56> concise readability, the corresponding immediate-mode concepts, widget APIs,57> ID stack mechanics, clipper usage, and styling rules translate directly 1-to-158> to C++. Guidelines apply both to core Dear ImGui and extension libraries, most59> importantly ImPlot.6061## 1. Naming Conventions: `build_gui` vs `render` / `draw`6263ImGui code does not render pixels to the screen directly. Instead, calling ImGui64functions builds and records **draw command lists** (vertex buffers, index65buffers, clip rects, and texture references) that are processed, rasterized,66and rendered later in the frame pipeline (via OpenGL, Vulkan, or67WebGL/WebViewer).6869- **Use `build_gui()` or `build_xxx_gui()`** (in Python) or `BuildGui()` /70 `BuildXxxGui()` (in C++) for all functions and methods that submit ImGui71 widgets (e.g., `build_control_panel_gui()`, `BuildControlPanelGui()`).72- **Avoid `render()` or `draw()`** for UI layout functions. Reserve `render` /73 `draw` strictly for low-level graphics backend operations (such as74 `ImGui_ImplOpenGL3_RenderDrawData`).7576```python77# Good: Accurately describes building the frame's UI hierarchy78def build_gui(self) -> None:79 self.build_header_gui()80 self.build_table_gui()8182# Avoid: Suggests immediate rasterization/rendering83def draw_ui(self):84 ...85```8687## 2. Decoupling GUI from Application Logic8889GUI code must strictly handle user layout, presentation, and input forwarding.90The GUI is intended solely to glue the underlying logic together, and that glue91layer should be as small and thin as possible. Never mix business logic,92network/RPC queries, file operations, or simulation steps directly into frame93build functions.9495- **Minimal Glue Layer**: The GUI component should do little more than bind96 user inputs to underlying library calls and render current state.97- **Separate Libraries**: Core business logic belongs in a standalone library98 or module (e.g., `inspector_lib.py` vs `inspector_gui.py`).99- **Asynchronous Execution**: Expensive operations (RPCs, subprocesses, large100 searches) must run on worker threads/tasks, with GUI code reading cached101 state or progress flags.102- **Testability**: Decoupled libraries can be thoroughly tested via standard103 unit tests without requiring an active ImGui context or window.104105## 3. Custom Widgets: Immediate Mode Pure Functions106107Custom widgets should remain lightweight, immediate-mode pure functions:108109- **Pure Immediate Mode**: Pass inputs as arguments, return output or mutation110 tuples (e.g., `changed, new_val = dial_widget(...)`). Avoid heavy stateful111 framework abstractions.112- **Avoid Per-Widget Persistent State**: Storing state across frames inside a113 widget incurs a maintenance burden and limits reusability across arbitrary114 contexts. Keep widgets stateless whenever possible; if state is essential,115 store it in the parent application state rather than inside the widget.116- **UX Customizations & Libraries**: Tailored visual controls (e.g., dial117 knobs, meters) are encouraged when they enhance UX. Instead of importing118 large external dependencies, copy and refactor small, focused snippets119 (must be MIT-licensed) into clean immediate-mode functions. In MuJoCo,120 shared custom ImGui widget functionality lives in the121 `imgui_widgets.h`/`cc` files in the Studio UX module.122- **Test Engine Readiness**: Expose interaction endpoints and bounding boxes123 using Dear ImGui Test Engine hooks/macros so automated test drivers can124 interact with custom controls.125- **Check Upstream & Changelog Before Implementing**: Before writing a new126 custom widget, layout behavior, or feature from scratch:127 - Check the latest upstream changelog for the `docking` branch at128 https://github.com/ocornut/imgui/blob/docking/docs/CHANGELOG.txt and129 inspect what has changed relative to the vendored version in MuJoCo.130 - Search Dear ImGui GitHub issues131 (https://github.com/ocornut/imgui/issues) and the Wiki132 (https://github.com/ocornut/imgui/wiki) for existing solutions, standard133 snippets, or discussions on the topic.134 - **Check Local vs Upstream (Update Dependency)**: In general, check both135 the local checked-out code and the latest code on GitHub. If the136 feature, widget, or bugfix already exists in a newer upstream release,137 the cleaner solution is often to update the vendored dependency rather138 than maintaining custom workarounds in application code.139140## 4. Disallowing Obsolete Functions & Internal Headers141142ImGui maintains a stable public API and clearly documents deprecations:143144- **Ban Obsolete Functions**: Always define `IMGUI_DISABLE_OBSOLETE_FUNCTIONS`145 (and `IMGUI_DISABLE_OBSOLETE_KEYIO`) during builds and framework updates to146 prevent deprecated patterns from lingering in the codebase.147- **Public API Stability**: Rely on the official public API (`imgui.h`). Avoid148 including internal headers (`imgui_internal.h`) in application and tool code149 unless implementing foundational backend extensions.150151## 5. Performance & Virtualization: `ImGuiListClipper`152153When rendering lists, tables, or log views that may contain many items (e.g.,154100+ bodies, simulation trajectories, log buffers):155156- **Always Virtualize with `ImGuiListClipper` / `imgui.ListClipper`**:157 Without clipping, ImGui generates vertices and text layouts for every item158 in the list even when offscreen. In remote streaming backends159 (`WebViewer`), unclipped tables saturate the WebSocket stream. In C++, use160 `ImGuiListClipper clipper; clipper.Begin(...)`; in Python, use161 `clipper = imgui.ListClipper(); clipper.Begin(...)`.162- **Clipper Usage**: The clipper calculates the visible range and only submits163 the items currently in view.164165```python166if imgui.BeginTable('ItemsTable', 1):167 clipper = imgui.ListClipper()168 clipper.Begin(len(items))169 while clipper.Step():170 for i in range(clipper.DisplayStart, clipper.DisplayEnd):171 imgui.TableNextRow()172 imgui.TableNextColumn()173 imgui.TextUnformatted(items[i].name)174 imgui.EndTable()175```176177- **Memory & Lifetime (`imgui.TextUnformatted`)**: Use `imgui.TextUnformatted`178 when displaying large or read-only text strings to avoid string copy and179 formatting overhead in Python bindings.180181## 6. Runtime IDs, the ID Stack, and Test Automation (`##` vs `###`)182183Dear ImGui identifies widgets by hashing their label text against the current ID184stack seed (`ImGuiWindow::GetID` calling `ImHashStr` in `imgui.cpp`).185Understanding the string operators `##` and `###` is critical for avoiding ID186collisions, maintaining state across dynamic label changes, and writing187automatable UIs.188189### Technical Mechanics: `##` vs `###`190191Both operators rely on Dear ImGui's internal text-rendering and hashing192routines:193194- **Display Boundary (`FindRenderedTextEnd`)**: Text rendering stops at the195 first occurrence of `##`. Because `###` also begins with `##`, **any196 characters following either `##` or `###` are hidden from the user**.197- **Hidden Disambiguator (`##`)**:198 - `ImHashStr` hashes the **entire string** (`"Label##Suffix"`), including199 both the visible label and the hidden suffix.200 - **Purpose**: Disambiguating widgets that share the same visible label201 within the same ID scope (e.g., multiple "Delete" buttons or repeating202 row entries).203 - **Why `Selectable(f'{body_name}##body_row_{i}')` uses `##`**: The204 visible label is `body_name`, but lists may contain duplicate body205 names or repeating entries. Appending `##body_row_{i}` hashes the full206 string `"{body_name}##body_row_{i}"`, producing a unique ID while207 keeping only `body_name` visible on screen. (Alternatively, wrap row208 rendering in `imgui.PushID(i)` / `imgui.PopID()`). If `###` were used209 here instead, `body_name` would be discarded completely from the hash.210- **ID Override / Seed Reset (`###`)**:211 - `ImHashStr` specifically detects three `#` characters (`###`). When212 encountered, it **resets the hash seed** back to the initial seed213 (`crc = seed`) and hashes **only** the substring following `###`. Any214 preceding visible text is completely discarded from the hash.215 - **Dynamic Labels**: When visible text changes across frames (e.g.,216 `f'Items ({count})###ItemCounterBtn'` or animated frame titles), `###`217 guarantees that the widget's internal ID remains constant. This218 preserves active focus, hover state, window docking positions, and219 collapse states.220 - **Semantically Meaningful Names for Automation**: For icon-only buttons221 (e.g., `f'{ICON_FA_PLAY}###PlayButton'`, where222 `ICON_FA_PLAY = '\uf04b'`), the icon glyph itself is constant across223 frames and would technically produce a stable hash even without `###`.224 However, without `###PlayButton`, the resulting ID is derived from an225 opaque unicode codepoint. Using `###PlayButton` assigns a226 human-readable, semantically meaningful identifier that facilitates227 agent-driven Dear ImGui Test Engine UI automation and computer control228 to reliably locate, inspect, and click the control by name229 (`"PlayButton"`).230231> [!NOTE]232> Dear ImGui Test Engine is not currently bundled with MuJoCo, but support is233> planned, along with recorded UI interaction testing.234235```python236# Icon glyph constant (e.g. FontAwesome play button unicode "\uf04b")237ICON_FA_PLAY = '\uf04b'238239# Semantic ID override: The glyph is displayed, but the ID hashes only240# 'PlayButton'. Provides a meaningful identifier for test automation agents and241# tooling.242imgui.Button(f'{ICON_FA_PLAY}###PlayButton')243244# Dynamic label with stable ID: Visible text changes with 'count', but the ID245# remains hashed from 'ItemCounterBtn', preserving focus, click, and hover246# state.247imgui.Button(f'Items ({count})###ItemCounterBtn')248249# Disambiguated duplicate label: Visible text is 'body_name', but the ID hashes250# the full string '{body_name}##body_row_{i}' to prevent ID collisions across251# rows.252imgui.Selectable(f'{body_name}##body_row_{i}', is_selected)253254# ID Scoping alternative: Using PushID / PopID for row loops instead of ##255# suffixes256imgui.PushID(i)257imgui.Selectable(body_name, is_selected)258imgui.PopID()259```260261### Further ImGui & ImPlot References262263Consult the upstream Dear ImGui and ImPlot sources and guides for authoritative264details and patterns:265266- **Dear ImGui ID Stack & API Guide**: `imgui.h` (around line 600) covers ID267 stack operations (`PushID`, `PopID`, `GetID`) and widget disambiguation268 rules.269- **Dear ImGui Programmer Guide & String Hashing**: The top of `imgui.cpp`270 contains the comprehensive Programmer Guide; lines ~2470–2515 contain271 `ImHashStr` documenting `###` seed resetting.272- **Dear ImGui FAQ**: The Dear ImGui [FAQ](https://github.com/ocornut/imgui/blob/master/docs/FAQ.md)273 includes dedicated sections on *"How can I have multiple widgets with the274 same label?"* and *"Why is my widget not reacting when I click on it?"*.275- **Interactive ImGui Demo Window**: `imgui_demo.cpp` provides complete,276 working examples of every widget, table feature, and clipper pattern. Call277 `imgui.ShowDemoWindow()` in your application to explore features278 interactively.279- **ImPlot Header & API Guide**: `implot.h` documents all plotting APIs,280 flags, styling properties, and context management.281- **Interactive ImPlot Demo Window**: `implot_demo.cpp` contains extensive282 interactive demos for lines, scatter, bars, shaded plots, and realtime283 scrolling subplots. Call `implot.ShowDemoWindow()` to inspect284 implementations.285286## 7. UI Design & Aesthetics: Grid Alignment (Tables API), Tooltips & Theming Costs287288High-quality ImGui tools balance immediate-mode simplicity with clean visual289structure and consistent styling:290291- **Grid-Aligned Widgets & Form Rows (Prefer the Tables API)**:292 - When arranging widgets into grids, key-value inspector panels, parameter293 sheets, or multi-row control bars where labels, dropdowns, or sliders294 must line up vertically and share equal widths, **prefer the modern295 Tables API** (`imgui.BeginTable`, `TableNextRow`, `TableNextColumn`).296 - Avoid legacy or brittle alignment techniques such as `imgui.Columns()`297 (obsolete), chaining `SameLine()` after variable-length text labels298 (different label widths shift every subsequent control horizontally),299 hardcoding pixel widths with `SetNextItemWidth(px)`, or inserting dummy300 spacing.301 - **Form & Multi-Control Alignment Recipe**:302 - Pass `int(imgui.TableFlags.SizingStretchProp) | int(imgui.TableFlags.NoPadOuterX)`303 for borderless layout tables so the outer left and right edges stay304 flush with surrounding non-table widgets.305 - Place text labels in `imgui.TableColumnFlags.WidthFixed` columns306 (omitting an explicit width so ImGui auto-fits to the widest label307 across all rows) and call `imgui.AlignTextToFramePadding()` before308 `TextUnformatted(...)` so label baselines center vertically against309 framed controls (`Combo`, `Slider`, `Input`).310 - Place controls (`Combo`, `SliderInt`, `SliderFloat`, `InputFloat`)311 in `imgui.TableColumnFlags.WidthStretch` columns using hidden `##`312 labels and `imgui.SetNextItemWidth(-1)` so every control in the313 column starts at the exact same X coordinate and stretches to the314 exact same length. If sliders also have trailing right-hand labels315 of different lengths (e.g., `'Quality'` vs `'Overlay'`), place those316 trailing labels in their own `WidthFixed` column—calling317 `SetNextItemWidth(-1)` directly on a widget with a visible label318 sizes the frame box to the full column width and clips the label.319320- **Tooltip Length & Aspect Ratio**:321 - Keep tooltips concise—avoid overly wordy descriptions that stretch into322 a wide single-line banner across the viewport.323 - Because `SetTooltip` and `SetItemTooltip` do not wrap text324 automatically, insert explicit newlines (`\n`) to keep the tooltip box325 at a pleasing, compact aspect ratio (~25–40 characters per line across326 2–3 lines) rather than a long horizontal strip.327 - Split lines at natural phrase, clause, or sentence boundaries (for328 example, between the primary action summary and secondary status or329 context) so each line reads cleanly on its own.330331- **Costs of Overriding Colors & Styles (Theming Impact)**:332 - Dear ImGui manages color palettes and visual metrics globally333 (supporting dark mode, light mode, or custom studio themes). Semantic334 colors are queried via `ImGui::GetStyleColorVec4(...)` in C++ or335 `imgui.GetStyleColorVec4(...)` in Python.336 - When custom widgets hardcode local color or style overrides (via337 `imgui.PushStyleColor` or `imgui.PushStyleVar`), **those values become338 decoupled from the global theme defaults**.339 - If an application or viewer switches themes (such as toggling between340 dark and light modes) or updates global styling metrics, hardcoded local341 overrides will not adapt and cannot be edited or updated via global342 defaults, often causing unreadable contrast or visual inconsistencies.343 - **Guideline**: Rely on semantic ImGui color tokens (`imgui.Col.Text`,344 `imgui.Col.FrameBg`, `imgui.Col.Button`, `imgui.Col.Header`, etc.) or345 derive dynamic colors via `imgui.GetStyleColorVec4(...)` rather than346 hardcoding static hex/RGB values. If a local override is strictly347 necessary, always pop it cleanly (`PopStyleColor` / `PopStyleVar`) and348 ensure it renders legibly across all supported themes.349350```python351# Good: Multi-row control bar with vertically aligned labels and equal-length352# dropdowns/sliders using a borderless layout table353flags = int(imgui.TableFlags.SizingStretchProp) | int(imgui.TableFlags.NoPadOuterX)354if imgui.BeginTable('##controls_layout', 4, flags):355 imgui.TableSetupColumn('Label', int(imgui.TableColumnFlags.WidthFixed))356 imgui.TableSetupColumn('Combo', int(imgui.TableColumnFlags.WidthStretch), 1.1)357 imgui.TableSetupColumn('Slider', int(imgui.TableColumnFlags.WidthStretch), 0.9)358 imgui.TableSetupColumn('TrailingLabel', int(imgui.TableColumnFlags.WidthFixed))359360 imgui.TableNextRow()361 imgui.TableNextColumn()362 imgui.AlignTextToFramePadding()363 imgui.TextUnformatted('Format:')364 imgui.TableNextColumn()365 imgui.SetNextItemWidth(-1)366 _, fmt_idx = imgui.Combo('##format', fmt_idx, ['Lossless', 'Lossy'])367 imgui.TableNextColumn()368 imgui.SetNextItemWidth(-1)369 _, quality = imgui.SliderInt('##quality', quality, 1, 100)370 imgui.TableNextColumn()371 imgui.AlignTextToFramePadding()372 imgui.TextUnformatted('Quality')373374 imgui.TableNextRow()375 imgui.TableNextColumn()376 imgui.AlignTextToFramePadding()377 imgui.TextUnformatted('Viewport:')378 imgui.TableNextColumn()379 imgui.SetNextItemWidth(-1)380 _, cam_idx = imgui.Combo('##camera', cam_idx, camera_names)381 imgui.TableNextColumn()382 imgui.SetNextItemWidth(-1)383 _, opacity = imgui.SliderFloat('##overlay', opacity, 0.0, 1.0)384 imgui.TableNextColumn()385 imgui.AlignTextToFramePadding()386 imgui.TextUnformatted('Overlay')387388 imgui.EndTable()389390# Avoid: Hardcoding colors detaches widgets from global themes391# (breaks dark/light modes)392# imgui.PushStyleColor(imgui.Col.Text, (0.1, 0.1, 0.1, 1.0)) # Unreadable in dark mode!393394# Good: Rely on semantic theme colors from ImGui395text_color = imgui.GetStyleColorVec4(imgui.Col.Text)396```397398## 8. Searchability of UI Strings in Source Code399400Developers and agents often search the codebase for text seen on screen to401locate the corresponding feature implementation:402403- **Single-Line Static Strings**: Keep user-visible label strings, button404 names, and headings on a single source line rather than breaking phrases405 across lines.406- **Conservative Interpolation**: Avoid over-fragmenting strings or splitting407 prefixes/suffixes across multiple format variables when a single cohesive408 string or format template makes the UI text searchable.409410## 9. Python Bindings411412- **Thin Binding Layer (No GUI/Application Logic in Bindings)**: Just as GUI413 code should avoid containing heavy application logic in glue layers,414 `pybind11` bindings (e.g., `ux.cc`) must remain thin wrappers around C++415 functions (such as `ux/*.h` and `ux/*.cc`). Do not write multi-widget ImGui416 layout or application logic directly inside Python binding lambdas;417 implement the widget or helper in C++ and bind the function instead.418- **Add On-Demand (Never Remove)**: Expose new ImGui or ImPlot bindings to419 Python only when they are actually needed and used by an application or420 plugin. Once added, do not remove them.421- **Order Parity**: When exposing new functions, add them in the exact order422 they appear in the upstream C++ headers to simplify future version upgrades423 and diffs.424- **Keyboard Capture Guarding (`io.WantCaptureKeyboard`)**: When implementing425 keyboard shortcuts (such as Space for pause or Backspace for reset), always426 check `not imgui.GetIO().WantCaptureKeyboard` before handling the key event.427 When a text input, filter box, or popup is active, ImGui sets428 `WantCaptureKeyboard = True`; ignoring this flag causes keystrokes typed429 into an input field to trigger simulation actions.430- **Initial Focus in Search & Popups (`SetKeyboardFocusHere`)**: When opening431 a command palette, filter box, or modal dialog, call432 `imgui.SetKeyboardFocusHere()` immediately before the `imgui.InputText` or433 `InputTextWithHint` call so the input field receives focus immediately434 without requiring a manual click.