Hir Grambarcodes Eval

hir-grambarcodes-eval

Gram Barcodes for Histopathology Tissue Texture Retrieval — Lifshitz et al. (2021) (arXiv:2111.15519, 2021)

What this evaluates

Evaluates histopathology image retrieval and classification performance using high-order texture features (Gram barcodes) extracted from CNN layers. It probes the model's ability to capture tissue texture patterns for accurate image matching and class prediction.

Datasets

• KimiaPath24 — total ?; splits: train (-1), test (-1)
• CRC — total ?; splits: 5-fold cross-validation (-1)
• EMC — total ?; splits: 5-fold cross-validation (-1)

Metrics

• η_total (primary) — range: percent
  • Overall retrieval accuracy score combining precision and recall components as defined in the paper.
• η_p — range: percent
  • Precision component of the retrieval score.
• η_w — range: percent
  • Recall/Weighted component of the retrieval score.
• Accuracy (primary) — range: percent
  • Percentage of correctly classified query images based on top-3 retrieval voting.

Input / output format

Input: Histopathology image passed through a pre-trained VGG19 network to extract Gram barcodes from specified convolutional layers.

Output: Predicted tissue class, determined by the mode of the classes of the top-3 most similar retrieved images (fallback to top-1 if no mode).

Scoring recipe

def predict_class(query_feat, db_feats, db_labels):
    sims = cosine_sim(query_feat, db_feats)
    top3_idx = argsort(sims, k=3, desc=True)
    top3_labels = [db_labels[i] for i in top3_idx]
    counts = Counter(top3_labels)
    if len(counts) == 1 or counts[top3_labels[0]] > 1:
        return top3_labels[0]
    else:
        return top3_labels[0]
def compute_accuracy(preds, gold):
    return sum(p == g for p, g in zip(preds, gold)) / len(gold)

Common pitfalls

• Uses retrieval-based voting (top-3 mode) instead of direct classification, which changes the evaluation dynamics compared to standard supervised benchmarks.
• Metrics differ across datasets: KimiaPath24 reports η_total/η_p/η_w, while CRC and EMC report standard accuracy with 5-fold CV, complicating cross-dataset comparison.

Evidence (verbatim from paper)

We achieved a top η_total score of 87.79% (η_p= 93.21%, η_w= 94.19%) on the provided training and testing sets using VGG19 layers ‘conv2_1’, ‘conv3_1’, and ‘conv5_1’ for Gram barcode generation. In our experiments, the predicted class of any query image is the mode of the classes of the top-3 most similar images retrieved by our algorithm. If there is no mode, the class of the most similar image (the top-1 image) is predicted to be the class of the query image.

Citation

@misc{lifshitz2021grambarcodes,
  title={Gram Barcodes for Histopathology Tissue Texture Retrieval},
  author={Lifshitz et al. (2021)},
  year={2021},
  note={arXiv:2111.15519}
}

• arXiv: 2111.15519