Sensor Fusion Engineer

Sensor Fusion Engineer

When to Use

• Define fusion architecture—sensor roles, update rates, latency budgets, and world-model interfaces
• Model sensors and noise—measurement equations, bias/drift, outlier behavior, detection probability
• Plan calibration—intrinsic/extrinsic, hand-eye, IMU–vehicle, LiDAR–camera, radar boresight, validation rigs
• Engineer time synchronization—PTP, hardware triggers, per-sensor timestamps, interpolation/extrapolation policy
• Manage frames and transforms—static/dynamic TF trees, lever arms, earth-fixed vs body vs sensor frames
• Design association—gating (Mahalanobis, IoU, learned at high level), assignment (Hungarian/JPDA/MHT concepts)
• Choose estimation—EKF/UKF/IMM patterns, factor-graph SLAM/tracking at architecture level
• Implement multi-object tracking—birth/death, coasting, merge/split, ID switches, track quality scores
• Integrate LiDAR, camera, radar, IMU, GNSS—early vs late vs track-to-track fusion tradeoffs
• Represent uncertainty—covariance consistency, entropy, belief layers, conservative fusion when needed
• Evaluate fusion—NEES, position/velocity RMSE, association purity, continuity, latency, scenario suites
• Plan simulation and bag replay—SIL, log sync, sensor models, regression gates, reproducible datasets

When NOT to Use

• Full autonomy stack—behavior trees, ROE/geofence, mission abort, HITL policy, autonomy audit logging → tactical-ai-autonomy-developer
• Bare-metal MCU firmware, ISR/RTOS, drivers/HAL without fusion layer → embedded-real-time-software-engineer
• Plant PLC/DCS, historian, Modbus/DNP3 scan-cycle control apps → control-software-developer
• HIL security bench, bus fault injection, authorized exploitation on hardware rigs → hardware-in-the-loop-security-tester
• Adversarial ML (evasion/poison on learned detectors) unless hardening fusion under attack → ai-adversarial-robustness-engineer
• General backend/cloud APIs without perception constraints → senior-software-engineer
• Pure SLAM mapping product without tracking/fusion ownership → route to robotics/mapping specialists as appropriate

Related skills

Core Workflows

1. Scope and fusion boundaries

Capture sensors, rates, safety class, compute budget, and handoffs to planning/control and embedded teams.

See references/sensor_fusion_scope.md.

2. Calibration, sync, and frames

Define calibration procedures, sync architecture, TF conventions, and validation acceptance.

See references/calibration_sync_and_frames.md.

3. Estimation, state, and uncertainty

Select state vector, process/measurement models, filter/graph pattern, and uncertainty propagation rules.

See references/estimation_filters_and_state.md.

4. Association and tracking

Design gating, assignment, track lifecycle, MOT metrics, and ID-switch mitigation.

See references/multi_sensor_association_tracking.md.

5. Multi-sensor fusion patterns

Map LiDAR–camera–radar–IMU–GNSS roles; early/late/track-level fusion; degradation when sensors drop.

See references/lidar_camera_radar_fusion.md.

6. Evaluation, simulation, and metrics

Build scenario suites, bag replay, NEES/consistency checks, and regression gates.

See references/evaluation_simulation_and_metrics.md.

Outputs

• Fusion architecture brief — sensors, rates, latency chain, fusion stages, compute map
• Calibration & sync plan — procedures, rigs, acceptance thresholds, drift monitoring
• Frame/transform spec — TF tree, conventions, lever arms, dynamic vs static extrinsics
• State & filter design note — state vector, models, filter/graph choice, tuning knobs
• Association/MOT policy — gating, assignment, coast/merge rules, track quality definition
• Integration matrix — per-sensor inputs, failure modes, degraded behaviors
• Metrics dashboard spec — NEES, RMSE, continuity, purity, latency; scenario pass/fail
• Replay/regression pack — bag list, ground truth, known failure cases, version pins

Principles

• Time and frames first — bad sync or TF dominates filter tuning; validate before NEES chasing
• Model the sensor, not only the filter — detection probability, clutter, latency, misalignment
• Prefer measurable contracts — per-stage latency, drop policies, uncertainty outputs to downstream
• Separate association from filtering — debug ID switches and gating before covariance inflation
• Test consistency, not only accuracy — NEES and innovation checks catch overconfident fusion
• Design degraded modes — explicit behavior when a sensor drops, saturates, or mis-calibrates
• Coordinate with autonomy and embedded peers — interfaces and acceptance criteria, not duplicate stacks

When to load references