Landcover AI Eval

landcover-ai-eval

LandCover.ai: Dataset for Automatic Mapping of Buildings, Woodlands, Water and Roads from Aerial Imagery — Boguszewski et al. (2020) (arXiv:2005.02264, 2020)

What this evaluates

Evaluates semantic segmentation models on high-resolution aerial imagery for mapping four land cover classes: buildings, woodlands, water, and roads. It probes the model's ability to accurately segment fine-grained, small, and narrow objects in rural environments from RGB imagery.

Datasets

• LandCover.ai — total 10674; splits: train (7470), val (1602), test (1602)

Metrics

• mIoU (primary) — range: percent
  • Average of Intersection over Union (IoU) across all classes. IoU is defined as the area of overlap between the ground truth and predicted class divided by the area of their union.

Input / output format

Input: 512×512 RGB image tiles extracted from aerial imagery.

Output: Pixel-wise semantic segmentation mask with 4 classes: buildings, woodlands, water, and roads.

Scoring recipe

ious = []
for cls in [buildings, woodlands, water, roads]:
    intersection = np.sum((pred == cls) & (gt == cls))
    union = np.sum((pred == cls) | (gt == cls))
    ious.append(intersection / union if union > 0 else 0.0)
mIoU = np.mean(ious) * 100

Common pitfalls

• Evaluation is performed on cropped 512×512 tiles with smaller edge tiles discarded, which may not reflect full-image inference performance or boundary context loss.
• Class area sizes are highly imbalanced; the authors applied custom loss weights during training, so evaluating without accounting for this imbalance can misrepresent capability on minority classes.
• The dataset covers only rural Polish terrain at 25–50 cm resolution, limiting generalizability to urban settings or other geographic regions.

Evidence (verbatim from paper)

Firstly we split 41 images and their corresponding masks into 512 × 512 tiles, getting rid of smaller ones (these on the right and bottom edges), and we shuffled them. Then we organized it as follows: 15% (1602) of tiles became test set, 15% (1602) of tiles became validation set, and the last 70% (7470) became train set. To evaluate the model, we use the mean intersection over union (mIoU), which is the standard metric for semantic segmentation. The mIoU is the average of intersection over union (IoU) across all classes. And IoU is defined as the area of overlap between the ground truth and predicted class divided by the area of their union.

Citation

@misc{boguszewski2020landcoverai,
  title={LandCover.ai: Dataset for Automatic Mapping of Buildings, Woodlands, Water and Roads from Aerial Imagery},
  author={Boguszewski et al. (2020)},
  year={2020},
  note={arXiv:2005.02264}
}

• arXiv: 2005.02264