Etide Eval

etide-eval

E-TIDE: Fast, Structure-Preserving Motion Forecasting from Event Sequences — Biswadeep Sen et al. (arXiv:2603.27757, 2026)

What this evaluates

Evaluates the ability of event-based motion forecasting models to predict future binary event occurrence maps (ON/OFF channels) from a sequence of past frames. It probes structural preservation of sparse motion traces, temporal consistency, and downstream utility for segmentation and tracking under varying traffic and high-speed motion regimes.

Datasets

• ETram — total ?; splits: train (-1), val (-1), test (-1)
• E-3DTrack — total ?; splits: train (-1), val (-1), test (-1)

Metrics

• MSE — range: other
  • Mean Squared Error between predicted logits and binary ground-truth occurrence maps. Lower is better.
• SSIM — range: [0, 1]
  • Structural Similarity Index measuring luminance, contrast, and structure similarity between predicted and ground-truth frames. Higher is better.
• LPIPS — range: [0, 1]
  • Learned Perceptual Image Patch Similarity computed using a pretrained feature network to measure perceptual distance. Lower is better.
• mIoU — range: [0, 1]
  • Mean Intersection over Union averaged across the ON and OFF polarity channels. Computed after binarizing logits with Otsu's threshold. Higher is better.
• aIoU (primary) — range: [0, 1]
  • Intersection over Union on the polarity-agnostic occupancy mask formed by OR-ing the ON and OFF channels. Computed after binarizing logits with Otsu's threshold. Higher is better.

Input / output format

Input: Sequence of $T_{in}=10$ frames, each represented as a two-channel binary occurrence map (ON/OFF) at 30 Hz. Spatial resolution is 128×128 (ETram uses random 512×512 activity crops downsampled; E-3DTrack uses full frame downsampled).

Output: Sequence of $T_{out}=10$ predicted frames as logits, which are sigmoided and binarized per-frame using Otsu's thresholding to yield binary ON/OFF event maps.

Scoring recipe

def score(predictions, ground_truth):
    # predictions: logits (T, 2, H, W), ground_truth: binary maps (T, 2, H, W)
    mse = np.mean((predictions - ground_truth)**2)
    ssim = compute_ssim(predictions, ground_truth)
    lpips = compute_lpips(predictions, ground_truth)
    
    all_on_ious, all_off_ious, all_a_ious = [], [], []
    for t in range(T):
        thresh = otsu_maximize_variance(predictions[t])
        pred_bin = (predictions[t] > thresh).astype(int)
        gt_bin = ground_truth[t]
        
        gt_bin[0])
        off_iou = jaccard_index(pred_bin[1], gt_bin[1])
        a_iou = jaccard_index(pred_bin[0] | pred_bin[1], gt_bin[0] | gt_bin[1])
        
        all_on_ious.append(on_iou)
        all_off_ious.append(off_iou)
        all_a_ious.append(a_iou)
        
    mIoU = np.mean(all_on_ious + all_off_ious)
    aIoU = np.mean(all_a_ious)
    return mse, ssim, lpips, mIoU, aIoU

Common pitfalls

• Thresholding protocol: The paper uses automatic Otsu thresholding per frame to binarize logits before computing IoU, rather than a fixed global threshold. Using a fixed threshold will yield different rankings.
• Input preprocessing mismatch: ETram requires random cropping of 512×512 activity regions before downsampling to 128×128, whereas E-3DTrack uses the full frame. Failing to replicate this exact preprocessing pipeline breaks comparability.

Evidence (verbatim from paper)

We evaluate prediction quality using pixel-level fidelity metrics (MSE↓, SSIM↑, LPIPS↓), as well as overlap metrics (mIoU (mean ON/OFF IoU), and aIoU (IoU on the polarity-agnostic occupancy mask formed by OR-ing both channels)). These overlap metrics also indicate downstream usability, as they reflect whether predicted event structure is suitable for segmentation-style evaluation. We apply a sigmoid to the predicted logits and binarize with a threshold τ, then compute IoU separately for ON and OFF channels against ground-truth occurrence maps. We report mIoU (mean of ON/OFF IoU) and aIoU. To avoid manual tuning, τ is selected automatically per frame via a thresholding rule that maximizes inter-class variance, and overlap metrics are accumulated globally over the test set.

Citation

@misc{sen2026etide,
  title={E-TIDE: Fast, Structure-Preserving Motion Forecasting from Event Sequences},
  author={Biswadeep Sen et al.},
  year={2026},
  note={arXiv:2603.27757}
}

• arXiv: 2603.27757