Lewmm Physical Eval

lewmm-physical-eval

LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels — Maes et al. (2026) (arXiv:2603.19312, 2026)

What this evaluates

Evaluates whether a latent world model captures physical structure and dynamics by probing latent representations for physical quantities and measuring predictive surprise under physical versus visual perturbations.

Datasets

• TwoRoom, PushT, OGBench-Cube, Reacher — total ?; splits: test (-1)

Metrics

• MSE (primary) — range: other
  • Mean squared error between predicted and ground-truth physical quantities (e.g., position, velocity). Lower is better.
• Pearson correlation (r) — range: other
  • Pearson correlation coefficient between predicted and ground-truth quantities. Higher is better.
• Surprise — range: other
  • Predictive loss or negative log-likelihood computed on perturbed trajectories relative to unperturbed baselines to measure violation of physical expectations.

Input / output format

Input: Latent embeddings extracted from context frames (for probing) or raw pixel trajectories with visual/physical perturbations (for VoE).

Output: Predicted physical quantities (e.g., 2D position, joint velocity, quaternion) or scalar surprise scores per timestep.

Scoring recipe

def compute_metrics(predictions, ground_truth):
    mse = np.mean((predictions - ground_truth) ** 2)
    r = pearsonr(predictions, ground_truth).statistic
    return mse, r
def compute_surprise(perturbed_loss, unperturbed_loss):
    return perturbed_loss - unperturbed_loss

Common pitfalls

• Probing metrics are reported separately for linear and non-linear (MLP) probes; results differ significantly and must not be conflated.
• Surprise signals are averaged over multiple perturbation types; physical vs. visual perturbations must be distinguished to assess physical understanding.
• Control evaluation uses fixed budgets and goal distances per environment; mixing these settings invalidates comparisons.

Evidence (verbatim from paper)

For each probe, we report the mean squared error (MSE) and the Pearson correlation coefficient between the predicted and ground-truth quantities.

Citation

@misc{maes2026lewmm,
  title={LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels},
  author={Maes et al. (2026)},
  year={2026},
  note={arXiv:2603.19312}
}

• arXiv: 2603.19312