Tusimple Eval

tusimple-eval

Ultra Fast Structure-aware Deep Lane Detection — Qin et al. (2020) (arXiv:2004.11757, 2020)

What this evaluates

Evaluates a model's ability to detect lane boundaries in highway driving scenarios using a point-based accuracy metric over predefined row anchors.

Datasets

• TuSimple — total 6408; splits: train (3268), validation (358), test (2782)

Metrics

• accuracy (primary) — range: percent
  • accuracy = (sum of correctly predicted lane points across all clips) / (sum of total ground truth lane points across all clips).

Input / output format

Input: RGB image (original resolution 1280x720, resized to 288x800 during training)

Output: Predicted lane points/coordinates via row-anchor classification or regression

Scoring recipe

total_correct = 0
total_gt = 0
for clip in dataset:
    gt_points = get_gt_lane_points(clip)
    pred_points = get_pred_lane_points(clip)
    correct = count_matched_points(gt_points, pred_points)
    total_correct += correct
    total_gt += len(gt_points)
accuracy = total_correct / total_gt

Common pitfalls

• The metric is point-based accuracy, not pixel-wise segmentation accuracy.
• Row anchors are fixed per dataset (TuSimple: 160-710, step 10; CULane: 260-530, step 10).
• Images are resized to 288x800 for training but evaluated at original resolution.

Evidence (verbatim from paper)

For TuSimple dataset, the main evaluation metric is accuracy. The accuracy is calculated by: accuracy = \sum_{clip}C_{clip} / \sum_{clip}S_{clip}, in which C_{clip} is the number of lane points predicted correctly and S_{clip} is the total number of ground truth in each clip.

Citation

@misc{qin2020ultrafast,
  title={Ultra Fast Structure-aware Deep Lane Detection},
  author={Qin et al. (2020)},
  year={2020},
  note={arXiv:2004.11757}
}

• arXiv: 2004.11757