# Geolocalization Eval

> Evaluates a model's ability to predict the geographic coordinates (latitude and longitude) of a query image from worldwide visual data. It probes fine-grained location-aware visual semantics and robustness to geographical heterogeneity across urban, regional, and continental scales. Use when the user wants to benchmark on IM2GPS3k, YFCC4K, or asks about evaluating this task. Reports threshold metric.

- Skill: `qhjqhj00/geolocalization-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/geolocalization-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/geolocalization-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/geolocalization-eval

---


# geolocalization-eval

> G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models — Pengyue Jia et al. (arXiv:2405.14702, 2024)

## What this evaluates

Evaluates a model's ability to predict the geographic coordinates (latitude and longitude) of a query image from worldwide visual data. It probes fine-grained location-aware visual semantics and robustness to geographical heterogeneity across urban, regional, and continental scales.

## Datasets

- **IM2GPS3k** — total ?; splits: test (-1)
- **YFCC4K** — total ?; splits: test (-1)

## Metrics

- `threshold metric` **(primary)** — range: percent
  - Percentage of predictions where the geodesic distance to the ground truth falls within specified thresholds (1km, 25km, 200km, 750km, and 2500km).

## Input / output format

**Input**: Query image (and optionally retrieved reference images/text for RAG prompts during inference).

**Output**: Predicted GPS coordinates (latitude, longitude).

## Scoring recipe

```python
def threshold_metric(predictions, ground_truths, thresholds=[1, 25, 200, 750, 2500]):
    results = {}
    for thresh_km in thresholds:
        correct = 0
        for pred, gt in zip(predictions, ground_truths):
            dist_km = geodesic_distance_meters(pred, gt) / 1000.0
            if dist_km <= thresh_km:
                correct += 1
        results[f'acc@{thresh_km}km'] = (correct / len(predictions)) * 100
    return results
```

## Common pitfalls

- Geodesic distance must be calculated using the specified Earth radius (6378137.0 m) to match the paper's results.
- The thresholds are evaluated independently, not cumulatively; accuracy at 25km is not derived from the 1km metric.
- Inference uses GPT-4V with temperature 1.2, introducing stochasticity; results should be averaged over multiple runs or fixed seeds for reproducibility.

## Evidence (verbatim from paper)

> Following previous work[cepeda2023geoclip] ; [zhou2024img2loc]*, we evaluate G3 with public datasets (IM2GPS3k*[hays2008im2gps]* and YFCC4K*[thomee2016yfcc100m]*) and a threshold metric. Given the predicted coordinates and the ground truths, this metric quantifies the percentage of predictions where the distance to the ground truth falls within specified thresholds (1km, 25km, 200km, 750km, and 2500km).

## Citation

```bibtex
@misc{jia2024g3,
  title={G3: An Effective and Adaptive Framework for Worldwide Geolocalization Using Large Multi-Modality Models},
  author={Pengyue Jia et al.},
  year={2024},
  note={arXiv:2405.14702}
}
```

- arXiv: 2405.14702