Table Row Detection Eval

table-row-detection-eval

Comparing Machine Learning Approaches for Table Recognition in Historical Register Books — Clinchant et al. (2019) (arXiv:1906.11901, 2019)

What this evaluates

Evaluates graph-based machine learning models for sequence labeling (BIESO) and table row detection on handwritten historical register books. The protocol tests the models' ability to segment table rows and label cell boundaries using pre-extracted textline and column features rather than raw images.

Datasets

• Dataset1 — total 144; splits: train (-1), test (-1)
• Dataset2 — total 150; splits: test (-1)

Metrics

• F1 score (primary) — range: [0, 1]
  • Harmonic mean of precision and recall: 2 * (precision * recall) / (precision + recall). Computed over predicted vs. ground-truth row labels or BIESO sequence tags.

Input / output format

Input: Graph representation where nodes are pre-extracted textlines or cells, and edges encode neighborhood relations (e.g., spatial proximity, column alignment).

Output: Sequence labels (BIESO tags) for cell boundary detection and binary/row labels indicating whether each textline belongs to a detected table row.

Scoring recipe

def compute_f1(predictions, gold):
    tp = sum(1 for p, g in zip(predictions, gold) if p == g and p == 1)
    fp = sum(1 for p, g in zip(predictions, gold) if p == 1 and g == 0)
    fn = sum(1 for p, g in zip(predictions, gold) if p == 0 and g == 1)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    return 2 * (precision * recall) / (precision + recall) if (precision + recall) > 0 else 0.0

Common pitfalls

• Relies entirely on pre-extracted features (textlines, columns) from an upstream IE workflow, so reported performance conflates table recognition with OCR/template matching quality.
• 4-fold cross-validation on Dataset1 may not reflect generalization to Dataset2, which comes from 15 different books with potentially different handwriting styles and layout variations.

Evidence (verbatim from paper)

both models achieve an 89 F1 score, demonstrating strong performance in row segmentation despite challenges like inconsistent handwriting, missing row separators, and variable cell alignment. Finally, we evaluated the CRF and GCN models both on the BIESO task, but also on the Table Row detection task using the workflow designed per se. For training and evaluating the algorithms, our dataset (hereafter named as dataset1) is composed of 144 manually annotated pages ... Training and testing have been performed using a 4-fold cross-validation.

Citation

@misc{clinchant2019table,
  title={Comparing Machine Learning Approaches for Table Recognition in Historical Register Books},
  author={Clinchant et al. (2019)},
  year={2019},
  note={arXiv:1906.11901}
}

• arXiv: 1906.11901