# Tabfact Eval

> Tests a model's ability to verify the truthfulness of a factual statement given a table. It probes structured reasoning capabilities by requiring the model to cross-reference table contents with a claim and output a binary label. Use when the user wants to benchmark on TabFact, or asks about evaluating this task. Reports binary classification accuracy.

- Skill: `qhjqhj00/tabfact-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/tabfact-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/tabfact-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/tabfact-eval

---


# tabfact-eval

> Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding — Wang et al. (2024) (arXiv:2401.04398, 2024)

## What this evaluates

Tests a model's ability to verify the truthfulness of a factual statement given a table. It probes structured reasoning capabilities by requiring the model to cross-reference table contents with a claim and output a binary label.

## Datasets

- **TabFact** — total ?; splits: test (-1)

## Metrics

- `binary classification accuracy` **(primary)** — range: [0, 1]
  - Percentage of correctly predicted truth labels (True/False) compared to the ground truth labels.

## Input / output format

**Input**: A tabular dataset and a factual statement.

**Output**: A binary label indicating whether the statement is True or False based on the table.

## Scoring recipe

```python
def score(pred, gold):
    return 1.0 if pred == gold else 0.0
accuracy = sum(score(p, g) for p, g in zip(predictions, golds)) / len(golds)
```

## Common pitfalls

- Accuracy is computed over the official test split; using training or validation splits inflates scores.
- The binary classification task requires strict adherence to the True/False labels without generating free-form explanations.

## Evidence (verbatim from paper)

> TabFact, on the other hand, is a table-based binary fact verification benchmark. The task is to ascertain the truthfulness of a given statement based on the table. ... and for TabFact, we employ the binary classification accuracy.

## Citation

```bibtex
@misc{wang2024chainoftable,
  title={Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding},
  author={Wang et al. (2024)},
  year={2024},
  note={arXiv:2401.04398}
}
```

- arXiv: 2401.04398