qhjqhj00 mobclip-downstream-eval
MobCLIP: Learning General-purpose Geospatial Representation at Scale — Wen et al. (2025) (arXiv:2506.01297, 2025)
What this evaluates
Evaluates geospatial representation models on 11 regression tasks spanning social, economic, and natural domains across multiple spatial scales (point, grid, county, city). It probes the model's ability to capture complex human-centric and environmental patterns using fused multimodal embeddings rather than relying solely on geographic coordinates.
Datasets
- MobCLIP Downstream Benchmark (11 tasks) — total ?; splits: test (-1); repo https://github.com/ylzhouchris/MobCLIP
Metrics
R^2(primary) — range: other (typically (-∞, 1])- Coefficient of determination measuring the proportion of variance in the target variable explained by the model. Calculated as 1 - (SS_res / SS_tot), where SS_res is the sum of squared residuals and SS_tot is the total sum of squares.
Input / output format
Input: Region-level embeddings (128-dimensional) derived from H3 grid cells. For point tasks, the embedding of the containing grid is used. For administrative tasks (county/city), the mean embedding of all constituent grids is computed. Baselines receive 2D coordinates.
Output: Continuous scalar prediction for each downstream task (e.g., population density, education years, energy consumption in tonnes of standard coal equivalent).
Scoring recipe
def compute_r2(y_true, y_pred):
y_mean = sum(y_true) / len(y_true)
ss_res = sum((y - yp)**2 for y, yp in zip(y_true, y_pred))
ss_tot = sum((y - y_mean)**2 for y in y_true)
return 1.0 - (ss_res / ss_tot)
# Report mean R^2 over 10 independent train/test runs with standard deviation
Common pitfalls
- Downstream models (LightGBM/Ridge) are trained on fixed embeddings without end-to-end fine-tuning, which may not fully leverage the representation quality.
- Two key datasets (ENE, COS) are proprietary and not publicly available, preventing exact reproduction of those specific task results.
- Baseline models are adapted to H3 grid centroids to match the spatial unit, potentially introducing coordinate misalignment not present in their original evaluations.
Evidence (verbatim from paper)
Model performances are measured by $R^{2}$, and the standard deviation over ten repeated trainings are marked in brackets below.
Citation
@misc{wen2025mobclip,
title={MobCLIP: Learning General-purpose Geospatial Representation at Scale},
author={Wen et al. (2025)},
year={2025},
note={arXiv:2506.01297}
}
- arXiv: 2506.01297