Cityintrusion Eval

cityintrusion-eval

PIDNet: An Efficient Network for Dynamic Pedestrian Intrusion Detection — Jingchen Sun et al. (arXiv:2009.00312, 2020)

What this evaluates

Evaluates real-time dynamic pedestrian intrusion detection from moving camera views, jointly performing area-of-interest segmentation and pedestrian detection to classify whether a pedestrian has intruded into a dynamic zone. It measures classification accuracy, segmentation quality, and detection precision while tracking computational efficiency.

Datasets

• Cityintrusion — total ?; splits: test (-1)
• Cityperson — total ?; splits: test (-1)
• Cityscape — total ?; splits: test (-1)

Metrics

• PID_Acc (primary) — range: percent
  • Percentage of correctly classified intrusion versus no-intrusion frames. Computed as the number of correct predictions divided by the total number of test samples.
• Seg IoU — range: [0, 1]
  • Intersection over Union between the predicted segmentation mask and the ground-truth mask for the area of interest. Calculated as the area of overlap divided by the area of union.
• Det AP — range: [0, 1]
  • Average Precision for pedestrian bounding box detection across varying recall thresholds, typically computed using the standard COCO-style integration over the precision-recall curve.
• PID_mAP — range: [0, 1]
  • Mean Average Precision for the intrusion detection task, aggregating AP scores across classes or thresholds as defined in Section 4.3 of the paper.

Input / output format

Input: Image frames resized to 512x1024 pixels.

Output: Binary intrusion classification label (intrusion/no-intrusion), segmentation mask for the dynamic area of interest, and bounding boxes with confidence scores for detected pedestrians.

Scoring recipe

def compute_pid_acc(pred_labels, true_labels):
    return sum(p == t for p, t in zip(pred_labels, true_labels)) / len(true_labels)

def compute_iou(pred_mask, true_mask):
    intersection = np.logical_and(pred_mask, true_mask).sum()
    union = np.logical_or(pred_mask, true_mask).sum()
    return intersection / union if union > 0 else 0.0

def compute_ap(pred_scores, pred_boxes, true_boxes, iou_thresh=0.5):
    # Standard AP calculation: sort by score, compute TP/FP at each threshold,
    # interpolate precision-recall curve, and integrate area under curve.
    pass

Common pitfalls

• Dynamic area-of-interest changes per frame, making static evaluation zones or fixed cropping invalid.
• Simply stacking existing segmentation and detection networks drastically increases parameters and reduces speed without improving accuracy.
• Lack of prior benchmarks for dynamic PID makes direct architectural comparisons difficult, requiring careful baseline selection.

Evidence (verbatim from paper)

PID_mAP and PID_Acc are the evaluation metrics as described in Section 4.3. The intersection over the union (IoU) is used as the evaluation metric of segmentation network, while the average precision (AP) is used as the evaluation metric of object detection.

Citation

@misc{sun2020pidnet,
  title={PIDNet: An Efficient Network for Dynamic Pedestrian Intrusion Detection},
  author={Jingchen Sun et al.},
  year={2020},
  note={arXiv:2009.00312}
}

• arXiv: 2009.00312