Spatiotemporal Kmeans Eval

spatiotemporal-kmeans-eval

Spatiotemporal k-means — Dorabiala et al. (2022) (arXiv:2211.05337, 2022)

What this evaluates

Evaluates clustering algorithms on their ability to track moving and static clusters in collective animal behavior data across space and time. It probes robustness in low-data regimes and the capacity to produce stable, interpretable cluster trajectories without relying on ground-truth labels for hyperparameter tuning.

Datasets

• Cakmak et al. Spatiotemporal Benchmark — total 1034; splits: test (1034)

Metrics

• total AMI (primary) — range: [0, 1]
  • Adjusted Mutual Information comparing the full cluster assignment history across all time steps against the ground truth static cluster labels.
• long-term AMI — range: [0, 1]
  • Adjusted Mutual Information comparing the refined, stable long-term cluster associations against the ground truth static cluster labels.

Input / output format

Input: Spatiotemporal datasets of N objects tracked over T time steps (size = N × T), containing spatial coordinates and temporal indices.

Output: Cluster assignment history for each time step, and optionally a refined set of stable, long-term cluster associations.

Scoring recipe

def compute_ami(predictions, ground_truth):
    # Assign unique labels to unassigned points to prevent AMI inflation
    pred_map = {p: i for i, p in enumerate(set(predictions))}
    gt_map = {g: i for i, g in enumerate(set(ground_truth))}
    pred_labels = [pred_map[p] for p in predictions]
    gt_labels = [gt_map[g] for g in ground_truth]
    return adjusted_mutual_info_score(pred_labels, gt_labels)

Common pitfalls

• Unassigned points must receive unique labels rather than a single shared label; otherwise, AMI is artificially inflated.
• Baseline methods allow point switching between time steps, but ground truth is static, creating a mismatch for short-term evaluation.
• Parameter tuning should not use ground truth labels, as this violates the unsupervised setting and inflates reported performance.

Evidence (verbatim from paper)

Cakmak et. al measure clustering quality with the adjusted mutual information (AMI) score and report execution time for a handful of baseline methods. The implemented baseline methods all allow points to switch clusters between time steps, and the reported AMI compares the full cluster assignment histories against the ground truth, which does not allow for point switching. While this comparison provides a way to evaluate the short-term associations, because of the mismatch between the method and the data generating mechanism, we believe that it is more informative to compare the stable, long-term associations derived from the full assignment histories against the ground truth. Therefore, we report both what refer to as the total AMI for the full cluster assignment histories and the long-term AMI for the long-term associations. We calculate total AMI as in Cakmak et. al by comparing the cluster assignment histories to the ground truth static cluster assignments.

Citation

@misc{dorabiala2022spatiotemporal,
  title={Spatiotemporal k-means},
  author={Dorabiala et al. (2022)},
  year={2022},
  note={arXiv:2211.05337}
}

• arXiv: 2211.05337