Lithology Classification Eval

lithology-classification-eval

GeoMind: An Agentic Workflow for Lithology Classification with Reasoned Tool Invocation — Zhou et al. (2026) (arXiv:2604.21501, 2026)

What this evaluates

Evaluates a model's ability to perform multi-class sequence labeling on multi-channel well-log time series data for lithology classification. It tests the model's capacity to handle diverse geological settings, manage distribution shifts, and produce stratigraphically plausible predictions.

Datasets

• SEAM — total 7092; splits: unspecified (-1)
• Facies — total 3164; splits: unspecified (-1)
• FORCE — total 52766; splits: unspecified (-1)
• GeoLink — total 580205; splits: unspecified (-1); repo https://github.com/LukasMosser/geolink_dataset

Metrics

• Weighted F1 (primary) — range: [0, 1]
  • Weighted average of per-class F1 scores, where each class's F1 is weighted by the number of true samples in that class. F1 for a class is the harmonic mean of its precision and recall.
• Weighted Precision — range: [0, 1]
  • Weighted average of per-class precision scores, weighted by class support (number of true samples).
• Weighted Recall — range: [0, 1]
  • Weighted average of per-class recall scores, weighted by class support.

Input / output format

Input: Depth-aligned multi-channel well-log time series sequences (e.g., gamma ray, resistivity, density) with corresponding lithology labels.

Output: Predicted lithology class label for each depth/sample point in the sequence.

Scoring recipe

def compute_weighted_f1(y_true, y_pred, num_classes):
    precisions, recalls, f1s, weights = [], [], [], []
    for c in range(num_classes):
        tp = np.sum((y_true == c) & (y_pred == c))
        fp = np.sum((y_true != c) & (y_pred == c))
        fn = np.sum((y_true == c) & (y_pred != c))
        prec = tp / (tp + fp) if (tp + fp) > 0 else 0.0
        rec = tp / (tp + fn) if (tp + fn) > 0 else 0.0
        f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0.0
        precisions.append(prec)
        recalls.append(rec)
        f1s.append(f1)
        weights.append(np.sum(y_true == c))
    return np.average(f1s, weights=weights)

Common pitfalls

• Using macro or micro averaging instead of the specified class-weighted average, which skews results on imbalanced geological datasets.
• Evaluating sequence labeling without accounting for stratigraphic fragmentation (salt-and-pepper noise), which is a key geological plausibility metric reported in the paper.
• Comparing models without aligning input preprocessing protocols, as the numerical module requires specific normalization and tool invocation steps.

Evidence (verbatim from paper)

To evaluate the lithology classification performances, here we select three widely used metrics, i.e., Precision, Recall, and F1 measure. Considering our task is a multi-class classification problem, we use the weighted average scores to evaluate the performances of our proposed methods and all baselines. Specifically, we weight the metrics of each class by the number of samples from that class.

Citation

@misc{zhou2026geomind,
  title={GeoMind: An Agentic Workflow for Lithology Classification with Reasoned Tool Invocation},
  author={Zhou et al. (2026)},
  year={2026},
  note={arXiv:2604.21501}
}

• arXiv: 2604.21501