Neural Stpp Eval

neural-stpp-eval

Neural Spatio-Temporal Point Processes — Chen et al. (2020) (arXiv:2011.04583, 2020)

What this evaluates

Evaluates the ability of spatio-temporal point process models to accurately capture complex, history-dependent spatial and temporal distributions of discrete events. It probes how well models can compute exact likelihoods for sequences of events in continuous space and time across diverse domains like seismology, epidemiology, and neuroscience.

Datasets

• PINWHEEL — total ?; splits: test (-1)
• EARTHQUAKES — total ?; splits: test (-1)
• COVID-19 CASES — total ?; splits: test (-1)
• BOLD5000 — total ?; splits: test (-1)

Metrics

• log-likelihood per event (primary) — range: other
  • Average log-likelihood of all events in a held-out test sequence. Computed separately for temporal and spatial components. Higher values indicate better model fit.

Input / output format

Input: Spatio-temporal event sequences consisting of timestamps and spatial coordinates, optionally conditioned on event history.

Output: Probability density/mass functions for next event location and time; evaluation reports aggregated log-likelihood scores rather than discrete predictions.

Scoring recipe

def compute_log_likelihood_per_event(model, test_sequences):
    total_ll = 0.0
    total_events = 0
    for seq in test_sequences:
        seq_ll = model.log_likelihood(seq)
        total_ll += seq_ll
        total_events += len(seq)
    return total_ll / total_events

Common pitfalls

• Log-likelihood is reported per event, not per sequence; comparing raw sequence LLs will misrepresent performance.
• Temporal and spatial log-likelihoods are reported separately; the joint likelihood is the sum of both.
• Values are negative; higher (closer to zero) is better, which is counterintuitive for some metrics.

Evidence (verbatim from paper)

Table 1: Log-likelihood per event on held-out test data (higher is better). Standard devs. estimated over 3 runs.

Citation

@misc{chen2020neuralstpp,
  title={Neural Spatio-Temporal Point Processes},
  author={Chen et al. (2020)},
  year={2020},
  note={arXiv:2011.04583}
}

• arXiv: 2011.04583