Salt Kg Eval

salt-kg-eval

SALT-KG: A Benchmark for Semantics-Aware Learning on Enterprise Tables — Mulang et al. (2026) (arXiv:2601.07638, 2026)

What this evaluates

Probes whether tabular models can effectively leverage declarative business knowledge and metadata semantics for prediction tasks, rather than relying solely on statistical correlations in raw features. It evaluates the impact of schema-grounded semantic embeddings on model inductive biases and relative performance across different model families.

Datasets

• SALT-KG — total ?; splits: test (-1); repo https://github.com/SAP-samples/salt-kg

Metrics

• ranking metrics (primary) — range: other
  • Standard tabular evaluation metrics used to rank model performance; exact formula not specified in the provided text.
• predictive accuracy — range: percent
  • Standard classification or regression accuracy; exact formula not specified in the provided text.

Input / output format

Input: Tabular row features (numerical and categorical) concatenated with a reduced-dimensional semantic representation derived from OBKG schema descriptors (CDS View, Fields, objNodeTypes) encoded via text-embedding-3-large and projected via PCA (16-64 components).

Output: Standard tabular prediction outputs (classification or regression labels/probabilities); exact format not specified.

Scoring recipe

# Standard tabular evaluation as implied by the text
def score(predictions, gold):
    # Compute standard accuracy or ranking metric
    accuracy = (predictions == gold).mean()
    return accuracy

Common pitfalls

• Semantic grounding modifies inductive biases rather than raw predictive accuracy, yielding only marginal gains in classical metrics.
• The dataset's relational scaffold lacks higher-order ontological depth (e.g., class hierarchies, transitivity), limiting semantic generalization.
• Neural baselines show greater sensitivity to semantic alignment than tree-based methods, which remain stable due to native handling of heterogeneous data.

Evidence (verbatim from paper)

Across all baselines, incorporating metadata-derived features rarely changes overall ranking metrics but consistently alters the relative performance of different model families, as shown by the slight deviations in Tab.[1].

Citation

@misc{mulang2026saltkg,
  title={SALT-KG: A Benchmark for Semantics-Aware Learning on Enterprise Tables},
  author={Mulang et al. (2026)},
  year={2026},
  note={arXiv:2601.07638}
}

• arXiv: 2601.07638