Tablevision Eval

tablevision-eval

TableVision: A Large-Scale Benchmark for Spatially Grounded Reasoning over Complex Hierarchical Tables — Chen et al. (2026) (arXiv:2604.03660, 2026)

What this evaluates

Probes multimodal large language models' ability to perform spatially grounded reasoning over complex hierarchical tables. It specifically evaluates performance degradation across three cognitive levels (Perception, Reasoning, Analysis) and measures how explicit spatial anchoring mitigates perceptual overload and spatial attention failure.

Datasets

• TableVision — total 6799; splits: test (1301), train (-1), val (-1)

Metrics

• exact-match Accuracy (%) (primary) — range: percent
  • Calculated as the percentage of instances where the model's predicted answer exactly matches the ground-truth answer, aggregated across 13 fine-grained task categories spanning L1 (Perception), L2 (Reasoning), and L3 (Analysis) cognitive levels.

Input / output format

Input: Image of a complex hierarchical table paired with a natural language query/question requiring spatial grounding and logical reasoning.

Output: Direct text answer (for zero-shot baseline evaluation) or structured chain-of-thought including bounding box coordinates followed by the final answer (for the proposed two-stage framework).

Scoring recipe

correct = 0
for pred, gold in zip(predictions, golds):
    if pred.strip().lower() == gold.strip().lower():
        correct += 1
accuracy = (correct / len(golds)) * 100

Common pitfalls

• Models often suffer from perceptual overload in dense tables, causing spatial attention failure and operating on incorrect semantic content.
• Zero-shot evaluation may mask underlying reasoning capabilities; providing ground-truth bounding boxes (Oracle probing) reveals a 20.7% accuracy surge, indicating spatial disorientation is the primary bottleneck rather than logical failure.
• Fine-tuning for complex L2/L3 tasks can cause performance regression on simple L1 perception tasks due to reasoning overhead and task interference.

Evidence (verbatim from paper)

We report the exact-match Accuracy (%) across the 13 fine-grained task categories spanning cognitive levels L1 to L3.

Citation

@misc{chen2026tablevision,
  title={TableVision: A Large-Scale Benchmark for Spatially Grounded Reasoning over Complex Hierarchical Tables},
  author={Chen et al. (2026)},
  year={2026},
  note={arXiv:2604.03660}
}

• arXiv: 2604.03660