Cnsight Eval

cnsight-eval

CNSight: Evaluation of Clinical Note Segmentation Tools — Surana et al. (2025) (arXiv:2512.22795, 2025)

What this evaluates

Evaluates the capability of models to detect section boundaries in clinical notes at the token level. It probes how well different architectures handle structured sentence-level segmentation versus unstructured freetext narrative variability in medical records.

Datasets

• MIMIC-IV Clinical Notes — total ?; splits: MIMIC Hospital Sentences (-1), MIMIC Hospital Freetext (-1)

Metrics

• Token-level F1 (primary) — range: percent
  • F1 = 2 * (Precision * Recall) / (Precision + Recall). Precision = TP / (TP + FP), Recall = TP / (TP + FN). TP is a predicted boundary token exactly matching a gold boundary; FP is a spurious split; FN is a missed gold boundary. Weighted F1 averages per-class performance proportional to class frequency, while micro-averaged F1 aggregates decisions across all boundaries.

Input / output format

Input: Tokenized clinical note text where each token is evaluated for boundary status.

Output: Binary token-level prediction sequence indicating whether each token is a section boundary (1) or not (0).

Scoring recipe

tp = sum(p == 1 and g == 1 for p, g in zip(predictions, gold))
fp = sum(p == 1 and g == 0 for p, g in zip(predictions, gold))
fn = sum(p == 0 and g == 1 for p, g in zip(predictions, gold))
precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0

Common pitfalls

• Applying macro-averaged F1 instead of the paper's specified weighted F1 for sentences or micro-averaged F1 for freetext, which misrepresents performance on imbalanced or highly variable section types.
• Evaluating at the span or sentence level rather than the token level, which changes the definition of true/false positives and invalidates the reported Precision/Recall/F1 scores.

Evidence (verbatim from paper)

Model performance is assessed using token-level Precision, Recall, and F1. For clinical note segmentation, we treat each predicted section boundary token as a classification decision. In this setup, a True Positive (TP) is a predicted boundary token that exactly matches a gold-standard boundary token, a False Positive (FP) is a predicted boundary token that does not correspond to any gold-standard boundary (i.e., a spurious split), and a False Negative (FN) is a gold-standard boundary token that the model fails to predict (i.e., a missed split).

Citation

@misc{surana2025cnsight,
  title={CNSight: Evaluation of Clinical Note Segmentation Tools},
  author={Surana et al. (2025)},
  year={2025},
  note={arXiv:2512.22795}
}

• arXiv: 2512.22795