etide-eval
E-TIDE: Fast, Structure-Preserving Motion Forecasting from Event Sequences — Biswadeep Sen et al. (arXiv:2603.27757, 2026)
What this evaluates
Evaluates the ability of event-based motion forecasting models to predict future binary event occurrence maps (ON/OFF channels) from a sequence of past frames. It probes structural preservation of sparse motion traces, temporal consistency, and downstream utility for segmentation and tracking under varying traffic and high-speed motion regimes.
Datasets
- ETram — total ?; splits: train (-1), val (-1), test (-1)
- E-3DTrack — total ?; splits: train (-1), val (-1), test (-1)
Metrics
MSE — range: other
- Mean Squared Error between predicted logits and binary ground-truth occurrence maps. Lower is better.
SSIM — range: [0, 1]
- Structural Similarity Index measuring luminance, contrast, and structure similarity between predicted and ground-truth frames. Higher is better.
LPIPS — range: [0, 1]
- Learned Perceptual Image Patch Similarity computed using a pretrained feature network to measure perceptual distance. Lower is better.
mIoU — range: [0, 1]
- Mean Intersection over Union averaged across the ON and OFF polarity channels. Computed after binarizing logits with Otsu's threshold. Higher is better.
aIoU (primary) — range: [0, 1]
- Intersection over Union on the polarity-agnostic occupancy mask formed by OR-ing the ON and OFF channels. Computed after binarizing logits with Otsu's threshold. Higher is better.
Input / output format
Input: Sequence of $T_{in}=10$ frames, each represented as a two-channel binary occurrence map (ON/OFF) at 30 Hz. Spatial resolution is 128×128 (ETram uses random 512×512 activity crops downsampled; E-3DTrack uses full frame downsampled).
Output: Sequence of $T_{out}=10$ predicted frames as logits, which are sigmoided and binarized per-frame using Otsu's thresholding to yield binary ON/OFF event maps.
Scoring recipe
def score(predictions, ground_truth):
# predictions: logits (T, 2, H, W), ground_truth: binary maps (T, 2, H, W)
mse = np.mean((predictions - ground_truth)**2)
ssim = compute_ssim(predictions, ground_truth)
lpips = compute_lpips(predictions, ground_truth)
all_on_ious, all_off_ious, all_a_ious = [], [], []
for t in range(T):
thresh = otsu_maximize_variance(predictions[t])
pred_bin = (predictions[t] > thresh).astype(int)
gt_bin = ground_truth[t]
gt_bin[0])
off_iou = jaccard_index(pred_bin[1], gt_bin[1])
a_iou = jaccard_index(pred_bin[0] | pred_bin[1], gt_bin[0] | gt_bin[1])
all_on_ious.append(on_iou)
all_off_ious.append(off_iou)
all_a_ious.append(a_iou)
mIoU = np.mean(all_on_ious + all_off_ious)
aIoU = np.mean(all_a_ious)
return mse, ssim, lpips, mIoU, aIoU
Common pitfalls
- Thresholding protocol: The paper uses automatic Otsu thresholding per frame to binarize logits before computing IoU, rather than a fixed global threshold. Using a fixed threshold will yield different rankings.
- Input preprocessing mismatch: ETram requires random cropping of 512×512 activity regions before downsampling to 128×128, whereas E-3DTrack uses the full frame. Failing to replicate this exact preprocessing pipeline breaks comparability.
Evidence (verbatim from paper)
We evaluate prediction quality using pixel-level fidelity metrics (MSE↓, SSIM↑, LPIPS↓), as well as overlap metrics (mIoU (mean ON/OFF IoU), and aIoU (IoU on the polarity-agnostic occupancy mask formed by OR-ing both channels)). These overlap metrics also indicate downstream usability, as they reflect whether predicted event structure is suitable for segmentation-style evaluation. We apply a sigmoid to the predicted logits and binarize with a threshold τ, then compute IoU separately for ON and OFF channels against ground-truth occurrence maps. We report mIoU (mean of ON/OFF IoU) and aIoU. To avoid manual tuning, τ is selected automatically per frame via a thresholding rule that maximizes inter-class variance, and overlap metrics are accumulated globally over the test set.
Citation
@misc{sen2026etide,
title={E-TIDE: Fast, Structure-Preserving Motion Forecasting from Event Sequences},
author={Biswadeep Sen et al.},
year={2026},
note={arXiv:2603.27757}
}
1---2name: etide-eval3description: Evaluates the ability of event-based motion forecasting models to predict future binary event occurrence maps (ON/OFF channels) from a sequence of past frames. It probes structural preservation of sparse motion traces, temporal consistency, and downstream utility for segmentation and tracking under varying traffic and high-speed motion regimes. Use when the user wants to benchmark on ETram, E-3DTrack, or asks about evaluating this task. Reports aIoU.4metadata:5 source: https://skillmd.com/skills/qhjqhj00/etide-eval6 skill_kind: dataset_eval7 source_arxiv: 2603.277578 bibtex_key: sen2026etide9 confidence: high10---1112# etide-eval1314> E-TIDE: Fast, Structure-Preserving Motion Forecasting from Event Sequences — Biswadeep Sen et al. (arXiv:2603.27757, 2026)1516## What this evaluates1718Evaluates the ability of event-based motion forecasting models to predict future binary event occurrence maps (ON/OFF channels) from a sequence of past frames. It probes structural preservation of sparse motion traces, temporal consistency, and downstream utility for segmentation and tracking under varying traffic and high-speed motion regimes.1920## Datasets2122- **ETram** — total ?; splits: train (-1), val (-1), test (-1)23- **E-3DTrack** — total ?; splits: train (-1), val (-1), test (-1)2425## Metrics2627- `MSE` — range: other28 - Mean Squared Error between predicted logits and binary ground-truth occurrence maps. Lower is better.29- `SSIM` — range: [0, 1]30 - Structural Similarity Index measuring luminance, contrast, and structure similarity between predicted and ground-truth frames. Higher is better.31- `LPIPS` — range: [0, 1]32 - Learned Perceptual Image Patch Similarity computed using a pretrained feature network to measure perceptual distance. Lower is better.33- `mIoU` — range: [0, 1]34 - Mean Intersection over Union averaged across the ON and OFF polarity channels. Computed after binarizing logits with Otsu's threshold. Higher is better.35- `aIoU` **(primary)** — range: [0, 1]36 - Intersection over Union on the polarity-agnostic occupancy mask formed by OR-ing the ON and OFF channels. Computed after binarizing logits with Otsu's threshold. Higher is better.3738## Input / output format3940**Input**: Sequence of $T_{in}=10$ frames, each represented as a two-channel binary occurrence map (ON/OFF) at 30 Hz. Spatial resolution is 128×128 (ETram uses random 512×512 activity crops downsampled; E-3DTrack uses full frame downsampled).4142**Output**: Sequence of $T_{out}=10$ predicted frames as logits, which are sigmoided and binarized per-frame using Otsu's thresholding to yield binary ON/OFF event maps.4344## Scoring recipe4546```python47def score(predictions, ground_truth):48 # predictions: logits (T, 2, H, W), ground_truth: binary maps (T, 2, H, W)49 mse = np.mean((predictions - ground_truth)**2)50 ssim = compute_ssim(predictions, ground_truth)51 lpips = compute_lpips(predictions, ground_truth)52 53 all_on_ious, all_off_ious, all_a_ious = [], [], []54 for t in range(T):55 thresh = otsu_maximize_variance(predictions[t])56 pred_bin = (predictions[t] > thresh).astype(int)57 gt_bin = ground_truth[t]58 59 on_iou = jaccard_index(pred_bin[0], gt_bin[0])60 off_iou = jaccard_index(pred_bin[1], gt_bin[1])61 a_iou = jaccard_index(pred_bin[0] | pred_bin[1], gt_bin[0] | gt_bin[1])62 63 all_on_ious.append(on_iou)64 all_off_ious.append(off_iou)65 all_a_ious.append(a_iou)66 67 mIoU = np.mean(all_on_ious + all_off_ious)68 aIoU = np.mean(all_a_ious)69 return mse, ssim, lpips, mIoU, aIoU70```7172## Common pitfalls7374- Thresholding protocol: The paper uses automatic Otsu thresholding per frame to binarize logits before computing IoU, rather than a fixed global threshold. Using a fixed threshold will yield different rankings.75- Input preprocessing mismatch: ETram requires random cropping of 512×512 activity regions before downsampling to 128×128, whereas E-3DTrack uses the full frame. Failing to replicate this exact preprocessing pipeline breaks comparability.7677## Evidence (verbatim from paper)7879> We evaluate prediction quality using pixel-level fidelity metrics (MSE↓, SSIM↑, LPIPS↓), as well as overlap metrics (mIoU (mean ON/OFF IoU), and aIoU (IoU on the polarity-agnostic occupancy mask formed by OR-ing both channels)). These overlap metrics also indicate downstream usability, as they reflect whether predicted event structure is suitable for segmentation-style evaluation. We apply a sigmoid to the predicted logits and binarize with a threshold τ, then compute IoU separately for ON and OFF channels against ground-truth occurrence maps. We report mIoU (mean of ON/OFF IoU) and aIoU. To avoid manual tuning, τ is selected automatically per frame via a thresholding rule that maximizes inter-class variance, and overlap metrics are accumulated globally over the test set.8081## Citation8283```bibtex84@misc{sen2026etide,85 title={E-TIDE: Fast, Structure-Preserving Motion Forecasting from Event Sequences},86 author={Biswadeep Sen et al.},87 year={2026},88 note={arXiv:2603.27757}89}90```9192- arXiv: 2603.27757