Culane Eval

culane-eval

Ultra Fast Structure-aware Deep Lane Detection — Qin et al. (2020) (arXiv:2004.11757, 2020)

What this evaluates

Evaluates lane detection performance in diverse urban and highway scenarios using an F1-measure based on IoU between predicted and ground truth lane lines.

Datasets

• CULane — total 133235; splits: train (88880), validation (9675), test (34680)

Metrics

• F1-measure (primary) — range: percent
  • F1 = 2 * Precision * Recall / (Precision + Recall). Each lane is treated as a 30-pixel-width line. IoU is computed between GT and prediction; IoU > 0.5 counts as True Positive.

Input / output format

Input: RGB image (original resolution 1640x590, resized to 288x800 during training)

Output: Predicted lane lines/coordinates via row-anchor classification or regression

Scoring recipe

TP = FP = FN = 0
for clip in dataset:
    gt_lanes = get_gt_lanes(clip)
    pred_lanes = get_pred_lanes(clip)
    for gt in gt_lanes:
        best_iou = max(iou(gt, pred) for pred in pred_lanes)
        if best_iou > 0.5:
            TP += 1
            break
    else:
        FN += 1
    FP += len(pred_lanes) - TP
Precision = TP / (TP + FP) if (TP + FP) > 0 else 0
Recall = TP / (TP + FN) if (TP + FN) > 0 else 0
F1 = 2 * Precision * Recall / (Precision + Recall)

Common pitfalls

• IoU threshold for positive match is strictly >0.5, not ≥0.5.
• Lanes are treated as 30-pixel-wide lines for IoU calculation, not 1-pixel curves.
• F1-measure is computed per clip, then averaged or summed across the dataset.

Evidence (verbatim from paper)

As for the evaluation metric of CULane, each lane is treated as a 30-pixel-width line. Then the intersection-over-union (IoU) is computed between ground truth and predictions. Predictions with IoUs larger than 0.5 are considered as true positives. F1-measure is taken as the evaluation metric and formulated as follows: F1-measure = 2PrecisionRecall/(Precision+Recall)

Citation

@misc{qin2020ultrafast,
  title={Ultra Fast Structure-aware Deep Lane Detection},
  author={Qin et al. (2020)},
  year={2020},
  note={arXiv:2004.11757}
}

• arXiv: 2004.11757