Coinfra Sync Eval

coinfra-sync-eval

CoInfra: A Large-Scale Cooperative Infrastructure Perception System and Dataset in Adverse Weather — Minghao Ning et al. (2025) (arXiv:2507.02245, 2025)

What this evaluates

Evaluates the temporal synchronization accuracy and robustness of a multi-node cooperative perception system under adverse weather and network conditions. It measures how well a delay-aware protocol aligns sensor data across nodes compared to naive asynchronous methods, focusing on timing errors, fusion completeness, and reaction latency.

Datasets

• CoInfra — total ?; splits: test (-1); repo https://github.com/NingMingHao/CoInfra

Metrics

• full_match_rate (primary) — range: [0, 1]
  • Percentage of synchronization cycles wherein data from all nodes successfully align to the global anchor. Computed as (number of fully aligned cycles / total cycles) * 100.
• reaction_time — range: other
  • Latency from the global anchor timestamp to the actual fusion trigger event timestamp. Measured in milliseconds.
• timing_error — range: other
  • Time difference between an individual node's data acquisition timestamp and its corresponding global time anchor. Measured in milliseconds.
• min_max_delay — range: other
  • Time difference between the earliest and latest data acquisitions among all nodes for each global trigger event. Measured in milliseconds.

Input / output format

Input: Timestamps of sensor data acquisitions from N nodes, global trigger schedule, and simulated/real network latency distributions (e.g., Gaussian delays, message drop rates).

Output: Per-cycle alignment status (aligned/misaligned), fusion trigger timestamps, and aggregated delay/error statistics across all synchronization cycles.

Scoring recipe

for each global trigger event:
  timestamps = [node_acq_time for node in nodes]
  timing_errors = [abs(t - global_anchor) for t in timestamps]
  min_max_delay = max(timestamps) - min(timestamps)
  full_match = 1 if all(t within window) else 0
  reaction_time = fusion_trigger_time - global_anchor

aggregate over all cycles:
  full_match_rate = sum(full_match) / total_cycles
  mean_timing_error = mean(timing_errors)
  mean_reaction_time = mean(reaction_time)
  mean_min_max_delay = mean(min_max_delay)

Common pitfalls

• Confusing the adaptive fusion window parameter (N sigma) with a measured metric; N sigma is a tunable configuration, not an outcome.
• Assuming late-arriving messages are discarded; the protocol explicitly retains them with reduced confidence or integrates them via post-fusion updates.
• Treating timing_error and min_max_delay as identical; timing_error measures per-node alignment to the anchor, while min_max_delay measures inter-node spread.

Evidence (verbatim from paper)

full match rate, defined as the percentage of synchronization cycles wherein data from all nodes successfully align to the global anchor; and reaction time, defined as the latency from the global anchor to the fusion trigger event.

Citation

@misc{ning2025coinfra,
  title={CoInfra: A Large-Scale Cooperative Infrastructure Perception System and Dataset in Adverse Weather},
  author={Minghao Ning et al. (2025)},
  year={2025},
  note={arXiv:2507.02245}
}

• arXiv: 2507.02245