Nwpu Resisc45 Eval

nwpu-resisc45-eval

Remote Sensing Image Scene Classification: Benchmark and State of the Art — Cheng et al. (2017) (arXiv:1703.00121, 2017)

What this evaluates

Evaluates the ability of image classification models to accurately categorize remote sensing scenes into one of 45 predefined land-use/land-cover categories. It probes robustness to realistic variations in spatial resolution, viewpoint, illumination, occlusion, and object pose that are common in aerial imagery.

Datasets

• NWPU-RESISC45 — total 31500; splits: train (-1), test (-1)

Metrics

• overall accuracy (primary) — range: [0, 1]
  • The proportion of correctly classified test samples out of the total number of test samples. Calculated as (number of correct predictions) / (total test samples).

Input / output format

Input: Remote sensing images of various geographic scenes.

Output: A single class label from the 45 predefined scene categories.

Scoring recipe

def compute_accuracy(predictions, labels):
    correct = sum(1 for p, l in zip(predictions, labels) if p == l)
    return correct / len(labels)

# Evaluation protocol:
# 1. Split dataset into train (10% or 20%) and test sets.
# 2. Train model on train split.
# 3. Predict labels for test set.
# 4. Compute overall accuracy using the function above.
# 5. Generate confusion matrix where entry (i,j) = rate of test samples from class i classified as class j.

Common pitfalls

• The evaluation uses fixed random training ratios (10% and 20%) rather than a single deterministic train/test split, meaning results can vary across different random seeds.
• Confusion matrices are only reported for the highest-accuracy method within each feature category, not for all baselines, which hides per-class error patterns for weaker models.

Evidence (verbatim from paper)

Tables 3-6 show the overall accuracies of three handcrafted global features, three unsupervised feature learning methods, three deep CNN features, and three fine-tuned CNN features, respectively, under the training ratios of 10% and 20%.

Citation

@misc{cheng2017remote,
  title={Remote Sensing Image Scene Classification: Benchmark and State of the Art},
  author={Cheng et al. (2017)},
  year={2017},
  note={arXiv:1703.00121}
}

• arXiv: 1703.00121